Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FileNotFoundException on executing ADD FILE #6389

Closed
3 of 4 tasks
jia-zhengwei opened this issue May 17, 2024 · 1 comment
Closed
3 of 4 tasks

FileNotFoundException on executing ADD FILE #6389

jia-zhengwei opened this issue May 17, 2024 · 1 comment
Labels
kind:bug This is a clearly a bug priority:major

Comments

@jia-zhengwei
Copy link

jia-zhengwei commented May 17, 2024 •
edited by pan3793

Code of Conduct

Search before asking

  • I have searched in the issues and found no similar issues.

Describe the bug

env:

  • hue.ini kyuubi part:
   options='{"url": "jdbc:hive2://xxx:2181,xxx:2181,xxx:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=kyuubi;", "driver": "org.apache.kyuubi.jdbc.KyuubiHiveDriver", "user": "", "password": ""}'
  • driver jar: kyuubi-hive-jdbc-shaded-1.7.1.jar
  • kyuubi version: 1.7.1
  • hue version: latest

errro msg:

 Error operating ExecuteStatement: java.io.FileNotFoundException: **File file:/mnt/disk1/yarn/nm-local-dir/usercache/xxx/appcache/application_1706840114147_88510/container_e10_1706840114147_88510_01_000001/; does not exist**

but actually, the folder(/mnt/disk1/yarn/nm-local-dir/usercache/xxx/appcache/application_1706840114147_88510/container_e10_1706840114147_88510_01_000001) exist.

Steps To Reproduce

  1. hue connect to kyuubi by zk
  2. login hue with ldap
  3. execute: ADD FILE "oss://xxx/path/to/xxx.py";

Affects Version(s)

1.7.1

Kyuubi Server Log Output

nothing error msg

Kyuubi Engine Log Output

nothing error msg

Kyuubi Server Configurations

defalut config

Kyuubi Engine Configurations

default config

Additional context

spark job logs:

An error occurred while calling o19.execute.

org.apache.kyuubi.jdbc.hive.KyuubiSQLException: org.apache.kyuubi.KyuubiSQLException: org.apache.kyuubi.KyuubiSQLException: Error operating ExecuteStatement: java.io.FileNotFoundException: **File file:/mnt/disk1/yarn/nm-local-dir/usercache/xxx/appcache/application_1706840114147_88510/container_e10_1706840114147_88510_01_000001/; does not exist**
 at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:666)
 at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:987)
 at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:656)
 at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:454)
 at org.apache.spark.SparkContext.addFile(SparkContext.scala:1620)
 at org.apache.spark.SparkContext.addFile(SparkContext.scala:1600)
 at org.apache.spark.sql.execution.command.AddFilesCommand.$anonfun$run$2(resources.scala:43)
 at org.apache.spark.sql.execution.command.AddFilesCommand.$anonfun$run$2$adapted(resources.scala:43)
 at scala.collection.immutable.Stream.foreach(Stream.scala:533)
 at org.apache.spark.sql.execution.command.AddFilesCommand.run(resources.scala:43)
 at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:75)
 at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:73)
 at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:84)
 at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.$anonfun$applyOrElse$1(QueryExecution.scala:98)
 at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$6(SQLExecution.scala:109)
 at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:169)
 at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:95)
 at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
 at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:64)
 at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:98)
 at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:94)
 at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:584)
 at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:176)
 at org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scala:584)
 at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDownWithPruning(LogicalPlan.scala:30)
 at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning(AnalysisHelper.scala:267)
 at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning$(AnalysisHelper.scala:263)
 at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:30)
 at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:30)
 at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:560)
 at org.apache.spark.sql.execution.QueryExecution.eagerlyExecuteCommands(QueryExecution.scala:94)
 at org.apache.spark.sql.execution.QueryExecution.commandExecuted$lzycompute(QueryExecution.scala:81)
 at org.apache.spark.sql.execution.QueryExecution.commandExecuted(QueryExecution.scala:79)
 at org.apache.spark.sql.Dataset.<init>(Dataset.scala:220)
 at org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:100)
 at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
 at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:97)
 at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:622)
 at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
 at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:617)
 at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.$anonfun$executeStatement$1(ExecuteStatement.scala:83)
 at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
 at org.apache.kyuubi.engine.spark.operation.SparkOperation.$anonfun$withLocalProperties$1(SparkOperation.scala:155)
 at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:169)
 at org.apache.kyuubi.engine.spark.operation.SparkOperation.withLocalProperties(SparkOperation.scala:139)
 at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.executeStatement(ExecuteStatement.scala:78)
 at org.apache.kyuubi.engine.spark.operation.ExecuteStatement$$anon$1.run(ExecuteStatement.scala:100)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:750)
 at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:69)
 at org.apache.kyuubi.engine.spark.operation.SparkOperation$$anonfun$onError$1.$anonfun$applyOrElse$1(SparkOperation.scala:189)
 at org.apache.kyuubi.Utils$.withLockRequired(Utils.scala:395)
 at org.apache.kyuubi.operation.AbstractOperation.withLockRequired(AbstractOperation.scala:51)
 at org.apache.kyuubi.engine.spark.operation.SparkOperation$$anonfun$onError$1.applyOrElse(SparkOperation.scala:177)
 at org.apache.kyuubi.engine.spark.operation.SparkOperation$$anonfun$onError$1.applyOrElse(SparkOperation.scala:172)
 at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:38)
 at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.$anonfun$executeStatement$1(ExecuteStatement.scala:88)
 at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
 at org.apache.kyuubi.engine.spark.operation.SparkOperation.$anonfun$withLocalProperties$1(SparkOperation.scala:155)
 at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:169)
 at org.apache.kyuubi.engine.spark.operation.SparkOperation.withLocalProperties(SparkOperation.scala:139)
 at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.executeStatement(ExecuteStatement.scala:78)
 at org.apache.kyuubi.engine.spark.operation.ExecuteStatement$$anon$1.run(ExecuteStatement.scala:100)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:750) Caused by: java.io.FileNotFoundException: File file:/mnt/disk1/yarn/nm-local-dir/usercache/xxx/appcache/application_1706840114147_88510/container_e10_1706840114147_88510_01_000001/; does not exist
 at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:666)
 at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:987)
 at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:656)
 at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:454)
 at org.apache.spark.SparkContext.addFile(SparkContext.scala:1620)
 at org.apache.spark.SparkContext.addFile(SparkContext.scala:1600)
 at org.apache.spark.sql.execution.command.AddFilesCommand.$anonfun$run$2(resources.scala:43)
 at org.apache.spark.sql.execution.command.AddFilesCommand.$anonfun$run$2$adapted(resources.scala:43)
 at scala.collection.immutable.Stream.foreach(Stream.scala:533)
 at org.apache.spark.sql.execution.command.AddFilesCommand.run(resources.scala:43)
 at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:75)
 at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:73)
 at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:84)
 at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.$anonfun$applyOrElse$1(QueryExecution.scala:98)
 at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$6(SQLExecution.scala:109)
 at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:169)
 at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:95)
 at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
 at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:64)
 at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:98)
 at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:94)
 at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:584)
 at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:176)
 at org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scala:584)
 at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDownWithPruning(LogicalPlan.scala:30)
 at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning(AnalysisHelper.scala:267)
 at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning$(AnalysisHelper.scala:263)
 at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:30)
 at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:30)
 at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:560)
 at org.apache.spark.sql.execution.QueryExecution.eagerlyExecuteCommands(QueryExecution.scala:94)
 at org.apache.spark.sql.execution.QueryExecution.commandExecuted$lzycompute(QueryExecution.scala:81)
 at org.apache.spark.sql.execution.QueryExecution.commandExecuted(QueryExecution.scala:79)
 at org.apache.spark.sql.Dataset.<init>(Dataset.scala:220)
 at org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:100)
 at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
 at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:97)
 at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:622)
 at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
 at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:617)
 at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.$anonfun$executeStatement$1(ExecuteStatement.scala:83) ... 11 more
 at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:69)
 at org.apache.kyuubi.operation.ExecuteStatement.waitStatementComplete(ExecuteStatement.scala:129)
 at org.apache.kyuubi.operation.ExecuteStatement.$anonfun$runInternal$1(ExecuteStatement.scala:161)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:750)
 at org.apache.kyuubi.jdbc.hive.KyuubiStatement.waitForOperationToComplete(KyuubiStatement.java:417)
 at org.apache.kyuubi.jdbc.hive.KyuubiStatement.executeWithConfOverlay(KyuubiStatement.java:200)
 at org.apache.kyuubi.jdbc.hive.KyuubiStatement.execute(KyuubiStatement.java:194)
 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.base/java.lang.reflect.Method.invoke(Method.java:566)
 at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
 at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374)
 at py4j.Gateway.invoke(Gateway.java:282)
 at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
 at py4j.commands.CallCommand.execute(CallCommand.java:79)
 at py4j.GatewayConnection.run(GatewayConnection.java:238)
 at java.base/java.lang.Thread.run(Thread.java:829)

Are you willing to submit PR?

  • Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
  • No. I cannot submit a PR at this time.
@jia-zhengwei jia-zhengwei added kind:bug This is a clearly a bug priority:major labels May 17, 2024
@pan3793 pan3793 changed the title hue do not support add files cmd when connnect to kyuubi by zk[Bug] FileNotFoundException on executing ADD FILE May 20, 2024
@jia-zhengwei
Copy link
Author

It's OK to use ADD FILE "oss://xxx/path/to/xxx.py" without semicolon

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind:bug This is a clearly a bug priority:major
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jia-zhengwei