Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error to route when requestAnalyzerConfig is True #487

Open
ndrluis opened this issue Sep 26, 2024 · 3 comments
Open

Error to route when requestAnalyzerConfig is True #487

ndrluis opened this issue Sep 26, 2024 · 3 comments

Comments

@ndrluis
Copy link
Member

ndrluis commented Sep 26, 2024

Hello,

I activated the io.trino.gateway.ha.module.QueryCountBasedRouterProvider and have two routing groups, each with one cluster.

I also have a rule that directs queries from the "dbt-trino" source to the large routing group. However, when I execute multiple queries, some of them are being routed to the adhoc routing group. Is this expected behavior, considering that the QueryCount router might have higher priority than the routing rule, or is this a bug?

Routing Rule

---
name: "dbt"
description: "if query from dbt"
condition: "request.getHeader(\"X-Trino-Source\").startsWith(\"dbt-trino\")"
actions:
  - "result.put(\"routingGroup\", \"large\")"

Clusters
image

Routing History
image (1)

@ndrluis
Copy link
Member Author

ndrluis commented Sep 26, 2024

I removed the QueryCountBasedRouterProvider, but the latest query from dbt is being routed to the adhoc group. The difference is that the latest query is a CREATE TABLE statement. I conducted 5 tests, and in all of them, the latest query was routed to adhoc.

I feel that it might just be a coincidence, because when I try to run just a CREATE TABLE, the query is routed to the large routing group.

@ndrluis
Copy link
Member Author

ndrluis commented Sep 26, 2024

Sometimes this error appears in the log

2024-09-26T22:50:44.831Z	ERROR	http-worker-62	io.trino.gateway.ha.router.RuleReloadingRoutingGroupSelector	Error opening rules configuration file, using routing group header as default.
com.google.common.base.VerifyException: Identifier cannot be empty or null
	at com.google.common.base.Verify.verify(Verify.java:126)
	at io.trino.sql.tree.Identifier.isValidIdentifier(Identifier.java:133)
	at io.trino.sql.tree.Identifier.<init>(Identifier.java:56)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
	at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:722)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:556)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:546)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:265)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:702)
	at io.trino.sql.tree.QualifiedName.of(QualifiedName.java:42)
	at io.trino.gateway.ha.router.TrinoQueryProperties.qualifyName(TrinoQueryProperties.java:386)
	at io.trino.gateway.ha.router.TrinoQueryProperties.getNames(TrinoQueryProperties.java:335)
	at io.trino.gateway.ha.router.TrinoQueryProperties.getNames(TrinoQueryProperties.java:341)
	at io.trino.gateway.ha.router.TrinoQueryProperties.getNames(TrinoQueryProperties.java:341)
	at io.trino.gateway.ha.router.TrinoQueryProperties.getNames(TrinoQueryProperties.java:341)
	at io.trino.gateway.ha.router.TrinoQueryProperties.getNames(TrinoQueryProperties.java:341)
	at io.trino.gateway.ha.router.TrinoQueryProperties.getNames(TrinoQueryProperties.java:341)
	at io.trino.gateway.ha.router.TrinoQueryProperties.getNames(TrinoQueryProperties.java:341)
	at io.trino.gateway.ha.router.TrinoQueryProperties.processRequestBody(TrinoQueryProperties.java:197)
	at io.trino.gateway.ha.router.TrinoQueryProperties.<init>(TrinoQueryProperties.java:140)
	at io.trino.gateway.ha.router.RuleReloadingRoutingGroupSelector.findRoutingGroup(RuleReloadingRoutingGroupSelector.java:99)
	at io.trino.gateway.ha.handler.RoutingTargetHandler.getBackendFromRoutingGroup(RoutingTargetHandler.java:86)
	at io.trino.gateway.ha.handler.RoutingTargetHandler.lambda$getRoutingDestination$0(RoutingTargetHandler.java:66)
	at java.base/java.util.Optional.orElseGet(Optional.java:364)
	at io.trino.gateway.ha.handler.RoutingTargetHandler.getRoutingDestination(RoutingTargetHandler.java:66)
	at io.trino.gateway.proxyserver.RouteToBackendResource.postHandler(RouteToBackendResource.java:67)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:146)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:189)
	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:159)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:93)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
	at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:274)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:266)
	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:253)
	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:696)
	at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:397)
	at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:349)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:358)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:312)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
	at org.eclipse.jetty.ee10.servlet.ServletHolder.handle(ServletHolder.java:736)
	at org.eclipse.jetty.ee10.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1614)
	at org.eclipse.jetty.ee10.servlet.ServletHandler$MappedServlet.handle(ServletHandler.java:1547)
	at org.eclipse.jetty.ee10.servlet.ServletChannel.dispatch(ServletChannel.java:824)
	at org.eclipse.jetty.ee10.servlet.ServletChannel.handle(ServletChannel.java:436)
	at org.eclipse.jetty.ee10.servlet.ServletHandler.handle(ServletHandler.java:464)
	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:597)
	at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:1060)
	at org.eclipse.jetty.server.Handler$Wrapper.handle(Handler.java:740)
	at org.eclipse.jetty.server.handler.EventsHandler.handle(EventsHandler.java:81)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:151)
	at org.eclipse.jetty.server.Handler$Wrapper.handle(Handler.java:740)
	at org.eclipse.jetty.server.handler.EventsHandler.handle(EventsHandler.java:81)
	at org.eclipse.jetty.server.Server.handle(Server.java:181)
	at org.eclipse.jetty.server.internal.HttpChannelState$HandlerInvoker.run(HttpChannelState.java:648)
	at org.eclipse.jetty.server.internal.HttpConnection.onFillable(HttpConnection.java:403)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:322)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:99)
	at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:478)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:441)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:293)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:201)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:311)
	at org.eclipse.jetty.util.thread.MonitoredQueuedThreadPool$1.run(MonitoredQueuedThreadPool.java:73)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:979)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1209)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1164)
	at java.base/java.lang.Thread.run(Thread.java:1570)

@ndrluis ndrluis changed the title QueryCountBasedRouterProvider Priority vs. Routing Rule Error to route when requestAnalyzerConfig is True Sep 30, 2024
@ndrluis
Copy link
Member Author

ndrluis commented Sep 30, 2024

I discovered that this error only happens when the requestAnalyzerConfig is set to True. Here is an example of the query:

CREATE OR REPLACE TABLE "catalog"."database"."example_table"

WITH (
      "extra_properties" = MAP(
        ARRAY['optimizer.enabled', 'compaction.enabled'],
        ARRAY['false', 'false']
      ))
AS (

WITH cte_example_sheets AS (
  SELECT *
  FROM "catalog"."database"."example_source"
)

, cte_example AS (
  SELECT
    CAST(field1 AS VARCHAR) AS alias1
    , CAST(field2 AS VARCHAR) AS alias2
    , CAST(field3 AS VARCHAR) AS alias3
    , CAST(SPLIT_PART(field4, '-', 1) AS VARCHAR) AS alias4
    , CAST(SPLIT_PART(field4, '-', 2) AS VARCHAR) AS alias5
    , CAST(field5 AS VARCHAR) AS alias6
    , CAST(DATE_PARSE(field6, '%Y-%m-%d %H:%i') AS TIMESTAMP(6)) AS alias7
    , CAST(DATE_PARSE(field7, '%Y-%m-%d %H:%i') AS TIMESTAMP(6)) AS alias8
    , CAST(DATE_PARSE(field8, '%Y-%m-%d %H:%i') AS TIMESTAMP(6)) AS alias9
    , CAST(DATE_PARSE(field9, '%Y-%m-%d %H:%i') AS TIMESTAMP(6)) AS alias10
    , CAST(field10 AS VARCHAR) AS alias11
    , CAST(field11 AS VARCHAR) AS alias12
    , CAST(field12 AS VARCHAR) AS alias13
  FROM cte_example_sheets
)

SELECT * FROM cte_example
)

The problem is that the exact same query runs without error when I use the Trino CLI as the client, but it returns an error every time I use the DBT client.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant