[Bug] CPU usage is high at irregular intervals (heap and profile information has been collected) #1273

hu3rror · 2024-05-18T07:05:28Z

Verify steps

I have read the documentation and understand the meaning of all configuration items I have written, avoiding a large number of seemingly useful options or default values.
I have not reviewed the documentation and resolve this issue.
I have not searched the Issue Tracker for the problem I am going to raise.
I have tested with the latest Alpha branch version, and the issue still persists.
I have provided server and client configuration files and processes that can reproduce the issue locally, rather than a desensitized complex client configuration file.
I have provided the simplest configuration that can reproduce the error I reported, rather than relying on remote servers, TUN, graphical client interfaces, or other closed-source software.
I have provided complete configuration files and logs, rather than providing only parts that I believe are useful due to confidence in my own intelligence.

Operating System

Linux

System Version

immortalWrt 23.05-SNAPSHOT r27694-e540dc745a / LuCI openwrt-23.05 branch git-24.092.44091-d24c2e3

Mihomo Version

Mihomo Meta alpha-fe88f0e linux amd64 with go1.22.2 Wed May 15 07:40:07 UTC 2024
Use tags: with_gvisor

profile.zip
heap.zip
config.yaml.zip

Configuration File

Please see [config.yaml.zip](https://github.com/MetaCubeX/mihomo/files/15360383/config.yaml.zip)

Description

The issue I'm facing is that mihomo consistently consumes 50% of the processing power of my dual-core, dual-thread (2C/2T) CPU, resulting in an average load of 1.00 across all cores.

Upon restarting mihomo (not killing it, just by API), the CPU usage normalizes to a consistent average load of 0.01 on each core.

Reproduction Steps

Unfortunately, this high CPU utilization problem recurs irregularly over time. Due to its sporadic nature, I regret that I cannot provide specific steps to reproduce it; however, I have activated debug mode and gathered heap and profile data during instances of elevated CPU usage.

Logs

No response

suchangv · 2024-05-25T17:06:48Z

windows 也遇到了

xianren78 · 2024-05-26T00:19:08Z

如果有tuic v5节点的话，参考这个commit加入cwnd=16配置试试。cc: @suchangv

suchangv · 2024-05-26T14:07:49Z

如果有tuic v5节点的话，参考这个commit加入cwnd=16配置试试。cc: @suchangv

不会搞，我试了多个版本的 mihomo 都有这个问题，发现之前用网线没问题，wifi 就有这个问题，协议是 Hysteria2

athenakia · 2024-05-27T17:53:33Z

如果有tuic v5节点的话，参考这个commit加入cwnd=16配置试试。cc: @suchangv

不会搞，我试了多个版本的 mihomo 都有这个问题，发现之前用网线没问题，wifi 就有这个问题，协议是 Hysteria2

你用的哪个版本？我这边之前也是类似情况，更新v1.18.4后貌似好了，你试试？

suchangv · 2024-05-28T15:31:56Z

如果有tuic v5节点的话，参考这个commit加入cwnd=16配置试试。cc: @suchangv

不会搞，我试了多个版本的 mihomo 都有这个问题，发现之前用网线没问题，wifi 就有这个问题，协议是 Hysteria2

你用的哪个版本？我这边之前也是类似情况，更新v1.18.4后貌似好了，你试试？

好像还真好了

c83a · 2024-05-29T03:54:34Z

/adapter/outboundgroup/groupbase.go 里的GetProxies函数调用过多的模式匹配
1 自己改配置文件把模式匹配全放进provider里，多创建几个provider,会好很多
2 让开发者更新GetProxies的实现，能根据Provider更新proxies数组缓存，而不是每次都建一个数组，每次都匹配一遍。。。
3 不会改的话，别用模式匹配

Skyxim · 2024-05-29T04:02:49Z

如果你 profile 是正确的，几乎所有 CPU 都消耗在了 H2，是否使用了相关协议的节点

athenakia · 2024-05-29T04:21:22Z

如果你 profile 是正确的，几乎所有 CPU 都消耗在了 H2，是否使用了相关协议的节点

我这边也是用hysteria2节点时遇到问题，配置文件很简单，关于节点的部分配置如下：

proxies:
  - name: 测试节点
    password: 88888888-8888-8888-8888-888888888888
    port: 8888
    server: example.com
    skip-cert-verify: false
    tfo: true
    type: hysteria2
    udp: true

athenakia · 2024-05-29T04:23:12Z

并且CPU占用是一阵一阵上升，通过任务管理器得性能监视可以看到波浪状，并不是一直爆满。更新到v1.18.4后暂时没有再出现问题。

c83a · 2024-05-30T01:09:30Z

From 4aeb993946781fc47017a0ccfec1a75c2c3a4623 Mon Sep 17 00:00:00 2001
From: c83a <c83a@git>
Date: Thu, 30 May 2024 08:49:58 +0800
Subject: [PATCH] Cached GetProxies

---
 adapter/outboundgroup/groupbase.go | 22 ++++++++++++++++++++++
 adapter/provider/provider.go       | 22 ++++++++++++++++++++--
 constant/provider/interface.go     |  5 ++++-
 3 files changed, 46 insertions(+), 3 deletions(-)

diff --git a/adapter/outboundgroup/groupbase.go b/adapter/outboundgroup/groupbase.go
index 69fbb617..3aca74a9 100644
--- a/adapter/outboundgroup/groupbase.go
+++ b/adapter/outboundgroup/groupbase.go
@@ -30,6 +30,7 @@ type GroupBase struct {
 	failedTime       time.Time
 	failedTesting    atomic.Bool
 	proxies          [][]C.Proxy
+	cached_proxies   []C.Proxy
 	versions         []atomic.Uint32
 	TestTimeout      int
 	maxFailedTimes   int
@@ -94,6 +95,27 @@ func (gb *GroupBase) Touch() {
 }
 
 func (gb *GroupBase) GetProxies(touch bool) []C.Proxy {
+	if touch {
+		for _, pd := range gb.providers {
+				pd.Touch()
+		}
+	}
+	if len(gb.cached_proxies)!= 0  {
+		return gb.cached_proxies
+	}
+	gb.cached_proxies = gb._GetProxies(false)
+	for _, pd := range gb.providers {
+		pd.AddFollower(gb)
+	}
+	return gb.cached_proxies
+
+}
+
+func (gb *GroupBase) DropCache() {
+	gb.cached_proxies = nil
+}
+
+func (gb *GroupBase) _GetProxies(touch bool) []C.Proxy {
 	var proxies []C.Proxy
 	if len(gb.filterRegs) == 0 {
 		for _, pd := range gb.providers {
diff --git a/adapter/provider/provider.go b/adapter/provider/provider.go
index daef017c..e589a5a1 100644
--- a/adapter/provider/provider.go
+++ b/adapter/provider/provider.go
@@ -9,6 +9,7 @@ import (
 	"runtime"
 	"strings"
 	"time"
+	"sync"
 
 	"github.com/metacubex/mihomo/adapter"
 	"github.com/metacubex/mihomo/common/convert"
@@ -27,7 +28,6 @@ import (
 const (
 	ReservedName = "default"
 )
-
 type ProxySchema struct {
 	Proxies []map[string]any `yaml:"proxies"`
 }
@@ -36,13 +36,14 @@ type ProxySchema struct {
 type ProxySetProvider struct {
 	*proxySetProvider
 }
-
 type proxySetProvider struct {
 	*resource.Fetcher[[]C.Proxy]
 	proxies          []C.Proxy
 	healthCheck      *HealthCheck
 	version          uint32
 	subscriptionInfo *SubscriptionInfo
+	locker		 sync.Mutex
+	follower	 map[types.AgroupBase]struct{}
 }
 
 func (pp *proxySetProvider) MarshalJSON() ([]byte, error) {
@@ -79,6 +80,7 @@ func (pp *proxySetProvider) Update() error {
 }
 
 func (pp *proxySetProvider) Initial() error {
+	pp.follower=make(map[types.AgroupBase]struct{})
 	elm, err := pp.Fetcher.Initial()
 	if err != nil {
 		return err
@@ -273,6 +275,9 @@ func (cp *compatibleProvider) RegisterHealthCheckTask(url string, expectedStatus
 	cp.healthCheck.registerHealthCheckTask(url, expectedStatus, filter, interval)
 }
 
+func (cp *compatibleProvider) AddFollower(gp types.AgroupBase) {
+}
+
 func stopCompatibleProvider(pd *CompatibleProvider) {
 	pd.healthCheck.close()
 }
@@ -297,11 +302,24 @@ func NewCompatibleProvider(name string, proxies []C.Proxy, hc *HealthCheck) (*Co
 	return wrapper, nil
 }
 
+func (pp *proxySetProvider)  AddFollower(gp types.AgroupBase) {
+	pp.locker.Lock()
+	defer pp.locker.Unlock()
+	pp.follower[gp]=struct{}{}
+}
+
+func (pp *proxySetProvider)  updateFollowerCache() {
+	for k:= range pp.follower{
+		k.DropCache()
+	}
+}
+
 func proxiesOnUpdate(pd *proxySetProvider) func([]C.Proxy) {
 	return func(elm []C.Proxy) {
 		pd.setProxies(elm)
 		pd.version += 1
 		pd.getSubscriptionInfo()
+		pd.updateFollowerCache()
 	}
 }
 
diff --git a/constant/provider/interface.go b/constant/provider/interface.go
index bb73d1bc..92f0ac99 100644
--- a/constant/provider/interface.go
+++ b/constant/provider/interface.go
@@ -54,7 +54,9 @@ func (pt ProviderType) String() string {
 		return "Unknown"
 	}
 }
-
+type AgroupBase interface{
+	DropCache()
+}
 // Provider interface
 type Provider interface {
 	Name() string
@@ -75,6 +77,7 @@ type ProxyProvider interface {
 	Version() uint32
 	RegisterHealthCheckTask(url string, expectedStatus utils.IntRanges[uint16], filter string, interval uint)
 	HealthCheckURL() string
+	AddFollower(AgroupBase)
 }
 
 // RuleProvider interface
-- 
2.42.0

试着写了下，仅供参考

c83a · 2024-05-30T02:05:23Z

From 1ce8f3a6bda55149963aec9194846a805f345419 Mon Sep 17 00:00:00 2001
From: c83a <c83a@git>
Date: Thu, 30 May 2024 09:57:32 +0800
Subject: [PATCH 2/2] fix inital provider follower

---
 adapter/provider/provider.go | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/adapter/provider/provider.go b/adapter/provider/provider.go
index e589a5a1..97a4d978 100644
--- a/adapter/provider/provider.go
+++ b/adapter/provider/provider.go
@@ -80,7 +80,7 @@ func (pp *proxySetProvider) Update() error {
 }
 
 func (pp *proxySetProvider) Initial() error {
-	pp.follower=make(map[types.AgroupBase]struct{})
+	pp.follower = nil
 	elm, err := pp.Fetcher.Initial()
 	if err != nil {
 		return err
@@ -305,7 +305,12 @@ func NewCompatibleProvider(name string, proxies []C.Proxy, hc *HealthCheck) (*Co
 func (pp *proxySetProvider)  AddFollower(gp types.AgroupBase) {
 	pp.locker.Lock()
 	defer pp.locker.Unlock()
-	pp.follower[gp]=struct{}{}
+	if len(pp.follower) != 0{
+		pp.follower[gp]=struct{}{}
+	}else{
+		pp.follower=make(map[types.AgroupBase]struct{})
+		pp.follower[gp]=struct{}{}
+	}
 }
 
 func (pp *proxySetProvider)  updateFollowerCache() {
-- 
2.42.0

修了下bug

Skyxim · 2024-05-30T02:14:36Z

并且CPU占用是一阵一阵上升，通过任务管理器得性能监视可以看到波浪状，并不是一直爆满。更新到v1.18.4后暂时没有再出现问题。

那暂时关闭此问题

c83a · 2024-05-30T02:55:57Z

From 46dae2e29cf7bafbf1266de6db2b9deae1168439 Mon Sep 17 00:00:00 2001
From: c83a <c83a@git>
Date: Thu, 30 May 2024 10:50:41 +0800
Subject: [PATCH 3/3] fix inital folloer

---
 adapter/provider/provider.go | 1 -
 1 file changed, 1 deletion(-)

diff --git a/adapter/provider/provider.go b/adapter/provider/provider.go
index 97a4d978..9af21366 100644
--- a/adapter/provider/provider.go
+++ b/adapter/provider/provider.go
@@ -80,7 +80,6 @@ func (pp *proxySetProvider) Update() error {
 }
 
 func (pp *proxySetProvider) Initial() error {
-	pp.follower = nil
 	elm, err := pp.Fetcher.Initial()
 	if err != nil {
 		return err
-- 
2.42.0

想了想，去掉了 pp.follower = nil
因为follower的实现不用在Initial里，自己初始化，自己make 和add
加了这一条反而有bug

c83a · 2024-05-30T04:54:27Z

From 7df6a4021ede436d57072b962c530b0c1fcc3055 Mon Sep 17 00:00:00 2001
From: c83a <c83a@git>
Date: Thu, 30 May 2024 12:30:32 +0800
Subject: [PATCH 4/4] fix cache

---
 adapter/outboundgroup/groupbase.go | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/adapter/outboundgroup/groupbase.go b/adapter/outboundgroup/groupbase.go
index 3aca74a9..bda7b1af 100644
--- a/adapter/outboundgroup/groupbase.go
+++ b/adapter/outboundgroup/groupbase.go
@@ -100,14 +100,16 @@ func (gb *GroupBase) GetProxies(touch bool) []C.Proxy {
 				pd.Touch()
 		}
 	}
-	if len(gb.cached_proxies)!= 0  {
-		return gb.cached_proxies
+	proxies := gb.cached_proxies
+	if len(proxies)!= 0  {
+		return proxies
 	}
-	gb.cached_proxies = gb._GetProxies(false)
+	proxies = gb._GetProxies(false)
+	gb.cached_proxies = proxies
 	for _, pd := range gb.providers {
 		pd.AddFollower(gb)
 	}
-	return gb.cached_proxies
+	return proxies
 
 }
 
-- 
2.42.0

这样应该不用加lock这样也能保证不会返回nil
slice浅复制代替lock
极端情况下可能gb.cached_proxies和provider不一致,因为这个函数调用太频繁了。可以写个timer 强制DropCache，可以补足。或者验证version，hash。怎么取舍看开发者和用户。

c83a · 2024-05-30T08:19:55Z

或者使用channel，能够避免管理锁的负担。然后provider，groupbase就不需要加锁了，而且更方便实现。我再改改。

c83a · 2024-05-30T09:39:29Z

From 26b820d0d8674e32b33b01b8c71e2718772da75f Mon Sep 17 00:00:00 2001
From: c83a <c83a@git>
Date: Thu, 30 May 2024 17:32:41 +0800
Subject: [PATCH 6/6] Cached GetProxies using channel

---
 adapter/outboundgroup/groupbase.go | 30 ++++++++++++++++++------------
 adapter/provider/provider.go       | 20 +++++++++++++-------
 constant/provider/interface.go     |  5 +----
 3 files changed, 32 insertions(+), 23 deletions(-)

diff --git a/adapter/outboundgroup/groupbase.go b/adapter/outboundgroup/groupbase.go
index bda7b1af..f94ae209 100644
--- a/adapter/outboundgroup/groupbase.go
+++ b/adapter/outboundgroup/groupbase.go
@@ -31,6 +31,7 @@ type GroupBase struct {
 	failedTesting    atomic.Bool
 	proxies          [][]C.Proxy
 	cached_proxies   []C.Proxy
+	dirtyCache       chan struct{}
 	versions         []atomic.Uint32
 	TestTimeout      int
 	maxFailedTimes   int
@@ -100,23 +101,28 @@ func (gb *GroupBase) GetProxies(touch bool) []C.Proxy {
 				pd.Touch()
 		}
 	}
-	proxies := gb.cached_proxies
-	if len(proxies)!= 0  {
-		return proxies
-	}
-	proxies = gb._GetProxies(false)
-	gb.cached_proxies = proxies
-	for _, pd := range gb.providers {
-		pd.AddFollower(gb)
+	var proxies []C.Proxy
+	if gb.dirtyCache != nil{
+		select{
+		case <- gb.dirtyCache:
+			proxies = gb._GetProxies(false)
+			gb.cached_proxies = proxies
+		default:
+			proxies = gb.cached_proxies
+
+		}
+	}else{
+		gb.dirtyCache = make(chan struct{},3)
+		for _, pd := range gb.providers {
+			pd.AddFollower(gb.dirtyCache)
+		}
+		proxies = gb._GetProxies(false)
+		gb.cached_proxies = proxies
 	}
 	return proxies
 
 }
 
-func (gb *GroupBase) DropCache() {
-	gb.cached_proxies = nil
-}
-
 func (gb *GroupBase) _GetProxies(touch bool) []C.Proxy {
 	var proxies []C.Proxy
 	if len(gb.filterRegs) == 0 {
diff --git a/adapter/provider/provider.go b/adapter/provider/provider.go
index 93010d36..940bb21f 100644
--- a/adapter/provider/provider.go
+++ b/adapter/provider/provider.go
@@ -43,7 +43,7 @@ type proxySetProvider struct {
 	version          uint32
 	subscriptionInfo *SubscriptionInfo
 	locker		 sync.Mutex
-	follower	 map[types.AgroupBase]struct{}
+	follower	 map[chan<- struct{}]struct{}
 }
 
 func (pp *proxySetProvider) MarshalJSON() ([]byte, error) {
@@ -274,7 +274,7 @@ func (cp *compatibleProvider) RegisterHealthCheckTask(url string, expectedStatus
 	cp.healthCheck.registerHealthCheckTask(url, expectedStatus, filter, interval)
 }
 
-func (cp *compatibleProvider) AddFollower(gp types.AgroupBase) {
+func (cp *compatibleProvider) AddFollower(ch chan<- struct{}) {
 }
 
 func stopCompatibleProvider(pd *CompatibleProvider) {
@@ -301,22 +301,28 @@ func NewCompatibleProvider(name string, proxies []C.Proxy, hc *HealthCheck) (*Co
 	return wrapper, nil
 }
 
-func (pp *proxySetProvider)  AddFollower(gp types.AgroupBase) {
+func (pp *proxySetProvider)  AddFollower(ch chan<- struct{}) {
 	pp.locker.Lock()
 	defer pp.locker.Unlock()
 	if len(pp.follower) != 0{
-		pp.follower[gp]=struct{}{}
+		pp.follower[ch]=struct{}{}
 	}else{
-		pp.follower=make(map[types.AgroupBase]struct{})
-		pp.follower[gp]=struct{}{}
+		pp.follower=make(map[chan<- struct{}] struct{})
+		pp.follower[ch]=struct{}{}
 	}
 }
 
 func (pp *proxySetProvider)  updateFollowerCache() {
 	pp.locker.Lock()
 	defer pp.locker.Unlock()
+	void := struct{}{}
 	for k:= range pp.follower{
-		k.DropCache()
+		select{
+		case k <- void:
+			//pass
+		default:
+			//pass
+		}
 	}
 
 }
diff --git a/constant/provider/interface.go b/constant/provider/interface.go
index 92f0ac99..fe48833f 100644
--- a/constant/provider/interface.go
+++ b/constant/provider/interface.go
@@ -54,9 +54,6 @@ func (pt ProviderType) String() string {
 		return "Unknown"
 	}
 }
-type AgroupBase interface{
-	DropCache()
-}
 // Provider interface
 type Provider interface {
 	Name() string
@@ -77,7 +74,7 @@ type ProxyProvider interface {
 	Version() uint32
 	RegisterHealthCheckTask(url string, expectedStatus utils.IntRanges[uint16], filter string, interval uint)
 	HealthCheckURL() string
-	AddFollower(AgroupBase)
+	AddFollower(chan <- struct{})
 }
 
 // RuleProvider interface
-- 
2.42.0

From 0fe026ad4cad97e8a9225c9cee700ac3007d54c0 Mon Sep 17 00:00:00 2001
From: c83a <c83a@git>
Date: Thu, 30 May 2024 14:16:26 +0800
Subject: [PATCH 5/6] updateFollowerCache add lock

---
 adapter/provider/provider.go | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/adapter/provider/provider.go b/adapter/provider/provider.go
index 9af21366..93010d36 100644
--- a/adapter/provider/provider.go
+++ b/adapter/provider/provider.go
@@ -313,9 +313,12 @@ func (pp *proxySetProvider)  AddFollower(gp types.AgroupBase) {
 }
 
 func (pp *proxySetProvider)  updateFollowerCache() {
+	pp.locker.Lock()
+	defer pp.locker.Unlock()
 	for k:= range pp.follower{
 		k.DropCache()
 	}
+
 }
 
 func proxiesOnUpdate(pd *proxySetProvider) func([]C.Proxy) {
-- 
2.42.0

主要是PATCH6/6
完成了channel部分改写

hu3rror added the bug Something isn't working label May 18, 2024

Skyxim closed this as completed May 30, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug] CPU usage is high at irregular intervals (heap and profile information has been collected) #1273

[Bug] CPU usage is high at irregular intervals (heap and profile information has been collected) #1273

hu3rror commented May 18, 2024 •

edited

suchangv commented May 25, 2024

xianren78 commented May 26, 2024 •

edited

suchangv commented May 26, 2024 •

edited

athenakia commented May 27, 2024

suchangv commented May 28, 2024

c83a commented May 29, 2024

Skyxim commented May 29, 2024

athenakia commented May 29, 2024

athenakia commented May 29, 2024

c83a commented May 30, 2024

c83a commented May 30, 2024

Skyxim commented May 30, 2024

c83a commented May 30, 2024

c83a commented May 30, 2024

c83a commented May 30, 2024

c83a commented May 30, 2024

[Bug] CPU usage is high at irregular intervals (heap and profile information has been collected) #1273

[Bug] CPU usage is high at irregular intervals (heap and profile information has been collected) #1273

Comments

hu3rror commented May 18, 2024 • edited

Verify steps

Operating System

System Version

Mihomo Version

Configuration File

Description

Reproduction Steps

Logs

suchangv commented May 25, 2024

xianren78 commented May 26, 2024 • edited

suchangv commented May 26, 2024 • edited

athenakia commented May 27, 2024

suchangv commented May 28, 2024

c83a commented May 29, 2024

Skyxim commented May 29, 2024

athenakia commented May 29, 2024

athenakia commented May 29, 2024

c83a commented May 30, 2024

c83a commented May 30, 2024

Skyxim commented May 30, 2024

c83a commented May 30, 2024

c83a commented May 30, 2024

c83a commented May 30, 2024

c83a commented May 30, 2024

hu3rror commented May 18, 2024 •

edited

xianren78 commented May 26, 2024 •

edited

suchangv commented May 26, 2024 •

edited