We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
(you don't have to strictly follow this form)
升级到0.4.0后 Daemon 开始报错,其他server worker 都读写正常
2024.05.17 11:54:09.007005 [ 36893 ] {} <Error> CnchRefreshMaterializedView: std::unordered_map<UUID, StorageID> DB::DaemonManager::getUUIDsFromCatalog(DB::DaemonManager::DaemonJobServerBGThread &): Code: 210, e.displayText() = DB::Exception: Unable to connect to HDFS: InvalidParameter: Cannot parse URI: hdfs://ns-fed, missing port or invalid HA configuration Caused by: HdfsConfigNotFound: Config key: dfs.ha.namenodes.ns-fed not found SQLSTATE: 08000, Stack trace (when copying this message, always include the lines below): 0. Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x26f45d32 in /data1/byconity/clickhouse 1. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x105d16e0 in /data1/byconity/clickhouse 2. DB::createHDFSFS(hdfsBuilder*) @ 0x20080bd2 in /data1/byconity/clickhouse 3. DB::HDFSFileSystem::HDFSFileSystem(DB::HDFSConnectionParams const&, int, int, int) @ 0x20072d4a in /data1/byconity/clickhouse 4. DB::DiskByteHDFS::DiskByteHDFS(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::HDFSConnectionParams const&) @ 0x20086882 in /data1/byconity/clickhouse 5. std::__1::shared_ptr<DB::DiskByteHDFS> std::__1::allocate_shared<DB::DiskByteHDFS, std::__1::allocator<DB::DiskByteHDFS>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, DB::HDFSConnectionParams&, void>(std::__1::allocator<DB::DiskByteHDFS> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, DB::HDFSConnectionParams&) @ 0x2008a2c7 in /data1/byconity/clickhouse 6. std::__1::shared_ptr<DB::IDisk> std::__1::__function::__policy_invoker<std::__1::shared_ptr<DB::IDisk> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::Context const>)>::__call_impl<std::__1::__function::__default_alloc_func<DB::registerDiskByteHDFS(DB::DiskFactory&)::$_0, std::__1::shared_ptr<DB::IDisk> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::Context const>)> >(std::__1::__function::__policy_storage const*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::Context const>&&) @ 0x200889e7 in /data1/byconity/clickhouse 7. DB::DiskFactory::create(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::Context const>) const @ 0x206fdf3f in /data1/byconity/clickhouse 8. DB::DiskSelector::DiskSelector(Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::Context const>) @ 0x20702bf2 in /data1/byconity/clickhouse 9. void std::__1::allocator<DB::DiskSelector>::construct<DB::DiskSelector, Poco::Util::AbstractConfiguration const&, char const* const&, std::__1::shared_ptr<DB::Context const> >(DB::DiskSelector*, Poco::Util::AbstractConfiguration const&, char const* const&, std::__1::shared_ptr<DB::Context const>&&) @ 0x208a0cd9 in /data1/byconity/clickhouse 10. DB::Context::getDiskSelector(std::__1::lock_guard<std::__1::mutex>&) const @ 0x20853b95 in /data1/byconity/clickhouse 11. DB::Context::getStoragePolicySelector(std::__1::lock_guard<std::__1::mutex>&) const @ 0x2083a21e in /data1/byconity/clickhouse 12. DB::Context::getStoragePolicy(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const @ 0x20847f8e in /data1/byconity/clickhouse 13. DB::StorageCnchMergeTree::getStoragePolicy(DB::IStorage::StorageLocation) const @ 0x217521ee in /data1/byconity/clickhouse 14. DB::StorageCnchMergeTree::StorageCnchMergeTree(DB::StorageID const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::StorageInMemoryMetadata const&, bool, std::__1::shared_ptr<DB::Context>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::MergeTreeMetaBase::MergingParams const&, std::__1::unique_ptr<DB::MergeTreeSettings, std::__1::default_delete<DB::MergeTreeSettings> >) @ 0x21751824 in /data1/byconity/clickhouse 15. DB::create(DB::StorageFactory::Arguments const&) @ 0x21e8a0ac in /data1/byconity/clickhouse 16. DB::StorageFactory::get(DB::ASTCreateQuery const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::Context>, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::ForeignKeysDescription const&, DB::UniqueNotEnforcedDescription const&, bool, std::__1::unique_ptr<DB::StorageFactory::HiveParams, std::__1::default_delete<DB::StorageFactory::HiveParams> >, bool) const @ 0x217aaead in /data1/byconity/clickhouse 17. DB::createStorageFromQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::Context const> const&) @ 0x1fe2fe7d in /data1/byconity/clickhouse 18. DB::Catalog::CatalogFactory::getTableByDefinition(std::__1::shared_ptr<DB::Context const> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x1fdab059 in /data1/byconity/clickhouse 19. DB::DaemonManager::constructStorageTrait(std::__1::shared_ptr<DB::Context>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x22c409be in /data1/byconity/clickhouse 20. DB::DaemonManager::getUUIDsFromCatalog(DB::DaemonManager::DaemonJobServerBGThread&) @ 0x22c3f420 in /data1/byconity/clickhouse 21. DB::DaemonManager::DaemonJobServerBGThread::executeImpl() @ 0x22c494c6 in /data1/byconity/clickhouse 22. DB::DaemonManager::DaemonJob::execute() @ 0x22c29103 in /data1/byconity/clickhouse 23. DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x20229ffe in /data1/byconity/clickhouse 24. DB::BackgroundSchedulePool::threadFunction() @ 0x2022c3a7 in /data1/byconity/clickhouse 25. void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, unsigned long, char const*, std::__1::shared_ptr<DB::CpuSet>)::$_1>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, unsigned long, char const*, std::__1::shared_ptr<DB::CpuSet>)::$_1&&)::'lambda'(), void ()> >(std::__1::__function::__policy_storage const*) @ 0x2022cbb7 in /data1/byconity/clickhouse 26. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x1060dc00 in /data1/byconity/clickhouse 27. void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda0'()> >(void*) @ 0x1061207a in /data1/byconity/clickhouse 28. start_thread @ 0x7e25 in /usr/lib64/libpthread-2.17.so 29. __clone @ 0xfebad in /usr/lib64/libc-2.17.so
0.4.0
The text was updated successfully, but these errors were encountered:
先执行 export LIBHDFS3_CONF='path to hdfs site' 在启动 dm
Sorry, something went wrong.
现在还有这个问题么?
@kevinthfang 0.4.0 版本还存在
Regarding this issue, I have already submitted a PR.
Merge pull request #1680 from frankye1982/issue_1674
eb49d53
sloved #1674 : An error occurred when the daemon was connected to hdfs ha
No branches or pull requests
(you don't have to strictly follow this form)
Bug Report
Briefly describe the bug
升级到0.4.0后 Daemon 开始报错,其他server worker 都读写正常
Version
0.4.0
The text was updated successfully, but these errors were encountered: