Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] java.lang.ClassNotFoundException: 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat #34944

Closed
3 tasks done
huangzhengshun opened this issue May 16, 2024 · 2 comments · Fixed by #34981
Closed
3 tasks done

[Bug] java.lang.ClassNotFoundException: 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat #34944

huangzhengshun opened this issue May 16, 2024 · 2 comments · Fixed by #34981

Comments

@huangzhengshun
Copy link

Search before asking

  • I had searched in the issues and found no similar issues.

Version

2.1.3

What's Wrong?

SemanticException java.lang.ClassNotFoundException: 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat
at org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:133)
at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeStatement(SQLQueryJob.java:582)
at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.lambda$1(SQLQueryJob.java:491)
at org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:189)
at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:498)
at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.extractData(SQLQueryJob.java:934)
at org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsContainer.readData(SQLEditor.java:3928)
at org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.lambda$0(ResultSetJobDataRead.java:123)
at org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:189)
at org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.run(ResultSetJobDataRead.java:121)
at org.jkiss.dbeaver.ui.controls.resultset.ResultSetViewer$ResultSetDataPumpJob.run(ResultSetViewer.java:5104)
at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:105)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: SemanticException java.lang.ClassNotFoundException: 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:256)
at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:242)
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:254)
at org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.execute(JDBCStatementImpl.java:330)
at org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:131)
... 12 more
Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: SemanticException java.lang.ClassNotFoundException: 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat
at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:335)
at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:199)
at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:260)
at org.apache.hive.service.cli.operation.Operation.run(Operation.java:247)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:541)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:527)
at sun.reflect.GeneratedMethodAccessor48.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)

What You Expected?

通过doris创建hive，能在hive中正常查询

How to Reproduce?

通过doris创建hive表，在doris中能正常查询，但是通过hive无法查询数据，同时获取表结构，发现OUTPUTFORMAT有问题，中间有\。
--创建后hive表结构
CREATE TABLE doris_test_v5(
account_book string COMMENT '',
voucher_no string COMMENT '',
item_no string COMMENT '',
voucher_type string COMMENT '',
account_no string COMMENT '',
subject_illustration string COMMENT '',
supplier_code string COMMENT '',
supplier_name string COMMENT '',
posting_code string COMMENT '',
special_gl_id string COMMENT '',
voucher_date string COMMENT '',
voucher_abstract string COMMENT '',
currency_code string COMMENT '',
original_amount string COMMENT '',
exchange_rate string COMMENT '',
equivalent_amount string COMMENT '',
quantity string COMMENT '',
unit string COMMENT '',
project_code string COMMENT '',
project_name string COMMENT '',
hirain1001 string COMMENT '',
cost_center string COMMENT '',
assigner_name string COMMENT '',
profit_center_code string COMMENT '',
profit_center string COMMENT '',
quality_cost string COMMENT '',
document_no string COMMENT '',
payment_terms string COMMENT '',
document_date string COMMENT '',
document_due_date string COMMENT '',
account_date string COMMENT '',
voucher_head_text string COMMENT '',
the_factory string COMMENT '',
movement_type string COMMENT '',
material_document string COMMENT '',
material_type string COMMENT '',
transaction_code string COMMENT '',
material_document_year string COMMENT '',
storage_location string COMMENT '',
the_batch string COMMENT '',
purchase_type string COMMENT '',
special_procurement string COMMENT '')
COMMENT ''
ROW FORMAT SERDE
'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat'
OUTPUTFORMAT
''org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'
LOCATION
'hdfs://slave2:7000/hive_data/D5D7ECADA9FF46AD9B4F695AE9FB39D3/41885DC9A4CE4BACA7212123C56F0DE3/615305DE04F749EA82AA1513CD4C28BD/doris_test_v5'
TBLPROPERTIES (
'doris.file_format'='parquet',
'doris.location'='hdfs://slave2:7000/hive_data/D5D7ECADA9FF46AD9B4F695AE9FB39D3/41885DC9A4CE4BACA7212123C56F0DE3/615305DE04F749EA82AA1513CD4C28BD/doris_test_v5',
'doris.version'='doris-2.1.3-rc06-e9f5388541',
'outputformat'='org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat',
'parquet.compression'='snappy',
'transient_lastDdlTime'='1715825319')

Anything Else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct
@ixzc
Copy link
Contributor

ixzc commented May 16, 2024 •
edited

we will fix it, and I will update pr here
#34981

@ixzc
Copy link
Contributor

ixzc commented May 16, 2024

Is doris version 2.1.3rc? Please provide the ddl, or you can add my wechat: Faitch_xzc

@wuwenchi wuwenchi mentioned this issue May 16, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[bugfix](hive)Misspelling of class names wuwenchi/doris_new
2 participants
@ixzc @huangzhengshun