Node not syncing - setting up a validator (synced state enabled)

[forexstudies]

Its been 24hrs and my node is not yet synched.
I have decided to setup a node with SYNC-STATE enabled (i.e. i am not downloading the entire chain). but after 24hrs later the command

chain-maind status 2>&1 | jq '.SyncInfo.catching_up'
true

still return true (i.e. not yet synched)
and i see the following errors in the log:
what could be the problem?

Jun 28 23:14:02 ip-172-31-40-50 chain-maind[6385]: 11:14PM ERR dialing failed (attempts: 1): auth failure: conn.ID (68b7b983b311943c85320c196100c2150477a29f) dialed ID (f1b442b5f4e6c95caa4bfe8c5d43e66b1aeb187b) mismatch addr={"id":"f1b442b5f4e6c95caa4bfe8c5d43e66b1aeb187b","ip":"162.55.103.217","port":26656} module=pex
Jun 28 23:14:31 ip-172-31-40-50 chain-maind[6385]: 11:14PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=9 numToDial=1
Jun 28 23:14:31 ip-172-31-40-50 chain-maind[6385]: 11:14PM INF Will dial address addr={"id":"a9238e5f1ee06077bb376f1fd74a818505836a24","ip":"35.241.129.98","port":26656} module=pex
Jun 28 23:14:31 ip-172-31-40-50 chain-maind[6385]: 11:14PM INF Dialing peer address={"id":"a9238e5f1ee06077bb376f1fd74a818505836a24","ip":"35.241.129.98","port":26656} module=p2p
Jun 28 23:14:32 ip-172-31-40-50 chain-maind[6385]: 11:14PM ERR dialing failed (attempts: 1): dial tcp 35.241.129.98:26656: i/o timeout addr={"id":"a9238e5f1ee06077bb376f1fd74a818505836a24","ip":"35.241.129.98","port":26656} module=pex
Jun 28 23:15:01 ip-172-31-40-50 chain-maind[6385]: 11:15PM INF Saving AddrBook to file book=/home/ubuntu/.chain-maind/config/addrbook.json module=p2p size=1055
Jun 28 23:15:01 ip-172-31-40-50 chain-maind[6385]: 11:15PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=9 numToDial=1
Jun 28 23:15:01 ip-172-31-40-50 chain-maind[6385]: 11:15PM INF Will dial address addr={"id":"c51507bc1980c5ea02f7ca83aaeb59cd9a5ca9f3","ip":"52.255.135.54","port":26656} module=pex
Jun 28 23:15:01 ip-172-31-40-50 chain-maind[6385]: 11:15PM INF Dialing peer address={"id":"c51507bc1980c5ea02f7ca83aaeb59cd9a5ca9f3","ip":"52.255.135.54","port":26656} module=p2p
Jun 28 23:15:02 ip-172-31-40-50 chain-maind[6385]: 11:15PM ERR dialing failed (attempts: 1): dial tcp 52.255.135.54:26656: i/o timeout addr={"id":"c51507bc1980c5ea02f7ca83aaeb59cd9a5ca9f3","ip":"52.255.135.54","port":26656} module=pex
Jun 28 23:15:31 ip-172-31-40-50 chain-maind[6385]: 11:15PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=9 numToDial=1
Jun 28 23:15:31 ip-172-31-40-50 chain-maind[6385]: 11:15PM INF Will dial address addr={"id":"789021ecd6fe442a3aae24367878c521a86abb4e","ip":"221.154.58.153","port":26656} module=pex
Jun 28 23:15:31 ip-172-31-40-50 chain-maind[6385]: 11:15PM INF Dialing peer address={"id":"789021ecd6fe442a3aae24367878c521a86abb4e","ip":"221.154.58.153","port":26656} module=p2p
Jun 28 23:15:31 ip-172-31-40-50 chain-maind[6385]: 11:15PM ERR dialing failed (attempts: 1): dial tcp 221.154.58.153:26656: connect: connection refused addr={"id":"789021ecd6fe442a3aae24367878c521a86abb4e","ip":"221.154.58.153","port":26656} module=pex
Jun 28 23:16:01 ip-172-31-40-50 chain-maind[6385]: 11:16PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=9 numToDial=1
Jun 28 23:16:01 ip-172-31-40-50 chain-maind[6385]: 11:16PM INF Will dial address addr={"id":"5591b621d834bf137b276e4021b0aaaac0abe386","ip":"54.38.137.222","port":26656} module=pex
Jun 28 23:16:01 ip-172-31-40-50 chain-maind[6385]: 11:16PM INF Dialing peer address={"id":"5591b621d834bf137b276e4021b0aaaac0abe386","ip":"54.38.137.222","port":26656} module=p2p
Jun 28 23:16:02 ip-172-31-40-50 chain-maind[6385]: 11:16PM ERR dialing failed (attempts: 2): dial tcp 54.38.137.222:26656: connect: connection refused addr={"id":"5591b621d834bf137b276e4021b0aaaac0abe386","ip":"54.38.137.222","port":26656} module=pex

[tomtau]

@forexstudies I assume you followed this tutorial: https://crypto.org/docs/getting-started/mainnet_validator.html#step-3-run-everything ?
Based on the provided log, it seems the node didn't even connect. Did it actually download the initial state (some early log entries should suggest that)? @zkaizhi @lezzokafka @aw126 may advise if this was discussed on Discord as well.

[forexstudies]

and this log is after running the reconfigure script and wiping all the data

Jun 30 09:20:46 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Starting Peer service impl="Peer{MConn{18.142.147.67:26656} d624d1c2f1051cbb4a744d819c82f9de1ffe6a23 out}" module=p2p peer={"id":"d624d1c2f1051cbb4a744d819c82f9de1ffe6a23","ip":"18.142.147.67","port":26656}
Jun 30 09:20:46 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Starting MConnection service impl=MConn{18.142.147.67:26656} module=p2p peer={"id":"d624d1c2f1051cbb4a744d819c82f9de1ffe6a23","ip":"18.142.147.67","port":26656}
Jun 30 09:20:46 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Added peer module=p2p peer={"Data":{},"Logger":{}}
Jun 30 09:20:46 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Connection is closed @ recvRoutine (likely by the other side) conn={"Logger":{}} module=p2p peer={"id":"d624d1c2f1051cbb4a744d819c82f9de1ffe6a23","ip":"18.142.147.67","port":26656}
Jun 30 09:20:46 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Stopping MConnection service impl={"Logger":{}} module=p2p peer={"id":"d624d1c2f1051cbb4a744d819c82f9de1ffe6a23","ip":"18.142.147.67","port":26656}
Jun 30 09:20:46 ip-172-31-40-50 chain-maind[14114]: 9:20AM ERR Stopping peer for error err=EOF module=p2p peer={"Data":{},"Logger":{}}
Jun 30 09:20:46 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Stopping Peer service impl={"Data":{},"Logger":{}} module=p2p peer={"id":"d624d1c2f1051cbb4a744d819c82f9de1ffe6a23","ip":"18.142.147.67","port":26656}
Jun 30 09:20:46 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Stopping gossipDataRoutine for peer module=consensus peer={"Data":{},"Logger":{}}
Jun 30 09:20:46 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Stopping gossipVotesRoutine for peer module=consensus peer={"Data":{},"Logger":{}}
Jun 30 09:20:46 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Stopping queryMaj23Routine for peer module=consensus peer={"Data":{},"Logger":{}}
Jun 30 09:20:46 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Stopping gossipVotesRoutine for peer module=consensus peer={"Data":{},"Logger":{}}
Jun 30 09:20:46 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Stopping gossipDataRoutine for peer module=consensus peer={"Data":{},"Logger":{}}
Jun 30 09:20:46 ip-172-31-40-50 chain-maind[14114]: 9:20AM ERR dialing failed (attempts: 1): auth failure: conn.ID (a00c6f650021e407feadff2260da7a2422ab0989) dialed ID (5a4074f93ad1d9ba4b2b688630ec291e1c5e26a2) mismatch addr={"id":"5a4074f93ad1d9ba4b2b688630ec291e1c5e26a2","ip":"89.58.34.100","port":26656} module=pex
Jun 30 09:20:46 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Fetching snapshot chunk chunk=99 format=1 height=6452000 module=statesync total=107
Jun 30 09:20:46 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Applied snapshot chunk to ABCI app chunk=95 format=1 height=6452000 module=statesync total=107
Jun 30 09:20:46 ip-172-31-40-50 chain-maind[14114]: 9:20AM ERR dialing failed (attempts: 1): dial tcp 135.181.22.146:26656: i/o timeout addr={"id":"291fa5a98d778fcd0b712b40cdc1db5562720823","ip":"135.181.22.146","port":26656} module=pex
Jun 30 09:20:46 ip-172-31-40-50 chain-maind[14114]: 9:20AM ERR dialing failed (attempts: 1): dial tcp 76.17.39.148:26643: i/o timeout addr={"id":"c282a9d3f775e6d9d59895503fdf3c234c669abd","ip":"76.17.39.148","port":26643} module=pex
Jun 30 09:20:48 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Stopping queryMaj23Routine for peer module=consensus peer={"Data":{},"Logger":{}}
Jun 30 09:20:48 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Fetching snapshot chunk chunk=100 format=1 height=6452000 module=statesync total=107
Jun 30 09:20:48 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Applied snapshot chunk to ABCI app chunk=96 format=1 height=6452000 module=statesync total=107
Jun 30 09:20:50 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Fetching snapshot chunk chunk=101 format=1 height=6452000 module=statesync total=107
Jun 30 09:20:50 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Applied snapshot chunk to ABCI app chunk=97 format=1 height=6452000 module=statesync total=107
Jun 30 09:20:52 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Fetching snapshot chunk chunk=102 format=1 height=6452000 module=statesync total=107
Jun 30 09:20:52 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Applied snapshot chunk to ABCI app chunk=98 format=1 height=6452000 module=statesync total=107
Jun 30 09:20:54 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Fetching snapshot chunk chunk=103 format=1 height=6452000 module=statesync total=107
Jun 30 09:20:54 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Applied snapshot chunk to ABCI app chunk=99 format=1 height=6452000 module=statesync total=107
Jun 30 09:20:56 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Fetching snapshot chunk chunk=104 format=1 height=6452000 module=statesync total=107
Jun 30 09:20:56 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Applied snapshot chunk to ABCI app chunk=100 format=1 height=6452000 module=statesync total=107
Jun 30 09:20:58 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Fetching snapshot chunk chunk=105 format=1 height=6452000 module=statesync total=107
Jun 30 09:20:58 ip-172-31-40-50 chain-maind[14114]: 9:20AM INF Applied snapshot chunk to ABCI app chunk=101 format=1 height=6452000 module=statesync total=107
Jun 30 09:21:00 ip-172-31-40-50 chain-maind[14114]: 9:21AM INF Fetching snapshot chunk chunk=106 format=1 height=6452000 module=statesync total=107
Jun 30 09:21:00 ip-172-31-40-50 chain-maind[14114]: 9:21AM INF Applied snapshot chunk to ABCI app chunk=102 format=1 height=6452000 module=statesync total=107
Jun 30 09:21:02 ip-172-31-40-50 chain-maind[14114]: 9:21AM INF Applied snapshot chunk to ABCI app chunk=103 format=1 height=6452000 module=statesync total=107
Jun 30 09:21:04 ip-172-31-40-50 chain-maind[14114]: 9:21AM INF Applied snapshot chunk to ABCI app chunk=104 format=1 height=6452000 module=statesync total=107
Jun 30 09:21:06 ip-172-31-40-50 chain-maind[14114]: 9:21AM INF Applied snapshot chunk to ABCI app chunk=105 format=1 height=6452000 module=statesync total=107
Jun 30 09:21:06 ip-172-31-40-50 chain-maind[14114]: 9:21AM ERR failed to restore snapshot err="import failed: tree must be empty"
Jun 30 09:21:06 ip-172-31-40-50 chain-maind[14114]: 9:21AM INF Applied snapshot chunk to ABCI app chunk=106 format=1 height=6452000 module=statesync total=107
Jun 30 09:21:06 ip-172-31-40-50 chain-maind[14114]: 9:21AM ERR State sync failed err="state sync aborted" module=statesync

[tomtau]

@forexstudies that looks better: at least based on the logs, it appears to be fetching snapshot chunks. I haven't seen that "import failed: tree must be empty" error before though.
I suggest hopping on https://discord.gg/pahqHz26q4 #technical-questions channel and see if someone faced a similar issue with that AWS AMI and statesync. One suggestion I saw was to downgrade to v3.3.2.
Once you get it resolved, you can post the answer here.

[forexstudies]

Ok i have posted on the discord but the following error output is still in the logs:

ubuntu@ip:~$ journalctl -u chain-maind | grep 'import failed'
Jun 28 06:07:59 ip-172-31-40-50 chain-maind[3029]: 6:07AM ERR failed to restore snapshot err="import failed: tree must be empty"
Jun 28 23:00:55 ip-172-31-40-50 chain-maind[6385]: 11:00PM ERR failed to restore snapshot err="import failed: tree must be empty"
Jun 30 08:55:33 ip-172-31-40-50 chain-maind[13746]: 8:55AM ERR failed to restore snapshot err="import failed: tree must be empty"
Jun 30 09:21:06 ip-172-31-40-50 chain-maind[14114]: 9:21AM ERR failed to restore snapshot err="import failed: tree must be empty"

last two lines are from today!

[forexstudies]

So I downgraded to version v3.3.2 it started off ok: and after 8hrs I got the error

Jul 05 03:15:14 ip-172-31-35-216 systemd[1]: Started chain-maind :: CRO Node (mars.crovalidator.au).
Jul 05 03:15:14 ip-172-31-35-216 chain-maind[2700]: 3:15AM INF starting ABCI with Tendermint
Jul 05 03:15:14 ip-172-31-35-216 chain-maind[2700]: 3:15AM INF Starting multiAppConn service impl=multiAppConn module=proxy
Jul 05 03:15:14 ip-172-31-35-216 chain-maind[2700]: 3:15AM INF Starting localClient service connection=query impl=localClient module=abci-client
Jul 05 03:15:14 ip-172-31-35-216 chain-maind[2700]: 3:15AM INF Starting localClient service connection=snapshot impl=localClient module=abci-client
Jul 05 03:15:14 ip-172-31-35-216 chain-maind[2700]: 3:15AM INF Starting localClient service connection=mempool impl=localClient module=abci-client
Jul 05 03:15:14 ip-172-31-35-216 chain-maind[2700]: 3:15AM INF Starting localClient service connection=consensus impl=localClient module=abci-client
Jul 05 03:15:14 ip-172-31-35-216 chain-maind[2700]: 3:15AM INF Starting EventBus service impl=EventBus module=events
Jul 05 03:15:14 ip-172-31-35-216 chain-maind[2700]: 3:15AM INF Starting PubSub service impl=PubSub module=pubsub
Jul 05 03:15:14 ip-172-31-35-216 chain-maind[2700]: 3:15AM INF Starting IndexerService service impl=IndexerService module=txindex
Jul 05 03:15:14 ip-172-31-35-216 chain-maind[2700]: 3:15AM INF Version info block=11 p2p=8 tendermint_version=0.34.14
Jul 05 03:15:14 ip-172-31-35-216 chain-maind[2700]: 3:15AM INF This node is not a validator addr=B9FB0F175CAF95B8DDBFFA03F684A916F9EB2539 module=consensus pubKey=eEW48sjKBMr5QGUVwElPmSPfypiENfpIrrGf00CbGHg=
Jul 05 03:15:14 ip-172-31-35-216 chain-maind[2700]: 3:15AM INF P2P Node ID ID=6421411fdd47bf8c78308ac6fd8cfa86824577ad file=/home/ubuntu/.chain-maind/config/node_key.json module=p2p
Jul 05 03:15:14 ip-172-31-35-216 chain-maind[2700]: 3:15AM INF Adding persistent peers addrs=["87c3adb7d8f649c51eebe0d3335d8f9e28c362f2@seed-0.crypto.org:26656","e1d7ff02b78044795371beb1cd5fb803f9389256@seed-1.crypto.org:26656","2c55809558a4e491e9995962e10c026eb9014655@seed-2.crypto.org:26656"] module=p2p
Jul 05 03:15:14 ip-172-31-35-216 chain-maind[2700]: 3:15AM INF Adding unconditional peer ids ids=[] module=p2p
Jul 05 03:15:14 ip-172-31-35-216 chain-maind[2700]: 3:15AM INF Add our address to book addr={"id":"6421411fdd47bf8c78308ac6fd8cfa86824577ad","ip":"0.0.0.0","port":26656}
 book=/home/ubuntu/.chain-maind/config/addrbook.json module=p2p

....

Jul 05 11:01:53 ip-172-31-35-216 chain-maind[2928]: 11:01AM ERR failed to restore snapshot err="import failed: tree must be empty"

and still processing but I dont know if this is going to finish, I trust that syncing should only take a couple of hours not days right! if it worth leaving it running for a couple of hours or this one is dead too? if its dead one suggestion from discord was to run with unsafe-reset-all, is this the correct command? chain-maind start --home /home/ubuntu/.chain-maind --unsafe-reset-all

Jul 05 13:04:24 ip-172-31-35-216 chain-maind[2928]: 1:04PM INF Saving AddrBook to file book=/home/ubuntu/.chain-maind/config/addrbook.json module=p2p size=1124
Jul 05 13:04:24 ip-172-31-35-216 chain-maind[2928]: 1:04PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=10 numToDial=0
Jul 05 13:04:54 ip-172-31-35-216 chain-maind[2928]: 1:04PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=10 numToDial=0
Jul 05 13:05:24 ip-172-31-35-216 chain-maind[2928]: 1:05PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=10 numToDial=0
Jul 05 13:05:54 ip-172-31-35-216 chain-maind[2928]: 1:05PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=10 numToDial=0
Jul 05 13:06:24 ip-172-31-35-216 chain-maind[2928]: 1:06PM INF Saving AddrBook to file book=/home/ubuntu/.chain-maind/config/addrbook.json module=p2p size=1124
Jul 05 13:06:24 ip-172-31-35-216 chain-maind[2928]: 1:06PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=10 numToDial=0
Jul 05 13:06:54 ip-172-31-35-216 chain-maind[2928]: 1:06PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=10 numToDial=0
Jul 05 13:07:24 ip-172-31-35-216 chain-maind[2928]: 1:07PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=10 numToDial=0
Jul 05 13:07:54 ip-172-31-35-216 chain-maind[2928]: 1:07PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=10 numToDial=0
Jul 05 13:08:24 ip-172-31-35-216 chain-maind[2928]: 1:08PM INF Saving AddrBook to file book=/home/ubuntu/.chain-maind/config/addrbook.json module=p2p size=1124
Jul 05 13:08:24 ip-172-31-35-216 chain-maind[2928]: 1:08PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=10 numToDial=0
Jul 05 13:08:54 ip-172-31-35-216 chain-maind[2928]: 1:08PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=10 numToDial=0
Jul 05 13:09:24 ip-172-31-35-216 chain-maind[2928]: 1:09PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=10 numToDial=0
Jul 05 13:09:54 ip-172-31-35-216 chain-maind[2928]: 1:09PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=10 numToDial=0
Jul 05 13:10:24 ip-172-31-35-216 chain-maind[2928]: 1:10PM INF Saving AddrBook to file book=/home/ubuntu/.chain-maind/config/addrbook.json module=p2p size=1124
Jul 05 13:10:24 ip-172-31-35-216 chain-maind[2928]: 1:10PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=10 numToDial=0

[tomtau]

the correct command is chain-maind unsafe-reset-all --home /home/ubuntu/.chain-maind assuming /home/ubuntu/.chain-maind is the home directory (I'm sure of that AMI details)

[forexstudies]

I posted on the discord group, didn't get any meaningful replies.
I have since terminated my instance several times and building again following the instructions to the letter...
i have no idea what it is doing?

the command

chain-maind status 2>&1 | jq '.SyncInfo.catching_up'

is still return true, even after more than 36hrs... how long does it take to join the network as a validator?

I am attaching the log here again:
chain.log

[tomtau]

usually within a few hours, I'd say. The log shows the same issue indicated before: #796 (reply in thread) Did you try using v3.3.2?
And did you clear up the storage with unsafe-reset-all? One other thing is: does the node have a public IP? and is this public IP + port set in the external-address field in the toml config?

[forexstudies]

yes I tried version 3.3.2 on a new AWS instance, and my instance has a public IP address, there are few address parameters where ports are opened. I will post the app.toml file and you can let me know which one:

# This is a TOML config file.
# For more information, see https://github.com/toml-lang/toml

###############################################################################
###                           Base Configuration                            ###
###############################################################################

# The minimum gas prices a validator is willing to accept for processing a
# transaction. A transaction's fees must meet the minimum of any denomination
# specified in this config (e.g. 0.25token1;0.0001token2).
minimum-gas-prices = "0.025basecro"

# default: the last 100 states are kept in addition to every 500th state; pruning at 10 block intervals
# nothing: all historic states will be saved, nothing will be deleted (i.e. archiving node)
# everything: all saved states will be deleted, storing only the current state; pruning at 10 block intervals
# custom: allow pruning options to be manually specified through 'pruning-keep-recent', 'pruning-keep-every', and 'pruning-interval'
pruning = "default"

# These are applied if and only if the pruning strategy is custom.
pruning-keep-recent = "0"
pruning-keep-every = "0"
pruning-interval = "0"

# HaltHeight contains a non-zero block height at which a node will gracefully
# halt and shutdown that can be used to assist upgrades and testing.
#
# Note: Commitment of state will be attempted on the corresponding block.
halt-height = 0

# HaltTime contains a non-zero minimum block time (in Unix seconds) at which
# a node will gracefully halt and shutdown that can be used to assist upgrades
# and testing.
#
# Note: Commitment of state will be attempted on the corresponding block.
halt-time = 0

# MinRetainBlocks defines the minimum block height offset from the current
# block being committed, such that all blocks past this offset are pruned
# from Tendermint. It is used as part of the process of determining the
# ResponseCommit.RetainHeight value during ABCI Commit. A value of 0 indicates
# that no blocks should be pruned.
#
# This configuration value is only responsible for pruning Tendermint blocks.
# It has no bearing on application state pruning which is determined by the
# "pruning-*" configurations.
#
# Note: Tendermint block pruning is dependant on this parameter in conunction
# with the unbonding (safety threshold) period, state pruning and state sync
# snapshot parameters to determine the correct minimum value of
# ResponseCommit.RetainHeight.
min-retain-blocks = 0

# InterBlockCache enables inter-block caching.
inter-block-cache = true

# IndexEvents defines the set of events in the form {eventType}.{attributeKey},
# which informs Tendermint what to index. If empty, all events will be indexed.
#
# Example:
# ["message.sender", "message.recipient"]
index-events = []

###############################################################################
###                         Telemetry Configuration                         ###
###############################################################################

[telemetry]

# Prefixed with keys to separate services.
service-name = ""

# Enabled enables the application telemetry functionality. When enabled,
# an in-memory sink is also enabled by default. Operators may also enabled
# other sinks such as Prometheus.
enabled = false

# Enable prefixing gauge values with hostname.
enable-hostname = false

# Enable adding hostname to labels.
enable-hostname-label = false

# Enable adding service to labels.
enable-service-label = false

# PrometheusRetentionTime, when positive, enables a Prometheus metrics sink.
prometheus-retention-time = 0

# GlobalLabels defines a global set of name/value label tuples applied to all
# metrics emitted using the wrapper functions defined in telemetry package.
#
# Example:
# [["chain_id", "cosmoshub-1"]]
global-labels = [
]

###############################################################################
###                           API Configuration                             ###
###############################################################################

[api]

# Enable defines if the API server should be enabled.
enable = false

# Swagger defines if swagger documentation should automatically be registered.
swagger = false

# Address defines the API server to listen on.
address = "tcp://0.0.0.0:1317"

# MaxOpenConnections defines the number of maximum open connections.
max-open-connections = 1000

# RPCReadTimeout defines the Tendermint RPC read timeout (in seconds).
rpc-read-timeout = 10

# RPCWriteTimeout defines the Tendermint RPC write timeout (in seconds).
rpc-write-timeout = 0

# RPCMaxBodyBytes defines the Tendermint maximum response body (in bytes).
rpc-max-body-bytes = 1000000

# EnableUnsafeCORS defines if CORS should be enabled (unsafe - use it at your own risk).
enabled-unsafe-cors = false

###############################################################################
###                           Rosetta Configuration                         ###
###############################################################################

[rosetta]

# Enable defines if the Rosetta API server should be enabled.
enable = false

# Address defines the Rosetta API server to listen on.
address = ":8080"

# Network defines the name of the blockchain that will be returned by Rosetta.
blockchain = "app"

# Network defines the name of the network that will be returned by Rosetta.
network = "network"

# Retries defines the number of retries when connecting to the node before failing.
retries = 3

# Offline defines if Rosetta server should run in offline mode.
offline = false

###############################################################################
###                           gRPC Configuration                            ###
###############################################################################

[grpc]

# Enable defines if the gRPC server should be enabled.
enable = true

# Address defines the gRPC server address to bind to.
address = "127.0.0.1:9090"

###############################################################################
###                        gRPC Web Configuration                           ###
###############################################################################

[grpc-web]

# GRPCWebEnable defines if the gRPC-web should be enabled.
# NOTE: gRPC must also be enabled, otherwise, this configuration is a no-op.
enable = true

# Address defines the gRPC-web server address to bind to.
address = "127.0.0.1:9091"

# EnableUnsafeCORS defines if CORS should be enabled (unsafe - use it at your own risk).
enable-unsafe-cors = false

###############################################################################
###                        State Sync Configuration                         ###
###############################################################################

# State sync snapshots allow other nodes to rapidly join the network without replaying historical
# blocks, instead downloading and applying a snapshot of the application state at a given height.
[state-sync]

# snapshot-interval specifies the block interval at which local state sync snapshots are
# taken (0 to disable). Must be a multiple of pruning-keep-every.
snapshot-interval = 0

# snapshot-keep-recent specifies the number of recent snapshots to keep and serve (0 to keep all).
snapshot-keep-recent = 2

[tomtau]

@forexstudies external-address is specified in config.toml, not app.toml, I think

[Dubersnack]

hey are you guys my node validators? if not can you guys help me I'm trying to withdraw my crypto its been sitting in my account for a while but I cant find it and don't know how to get it to verify on the explorer sites I know little to nothing about programing so explain so I can understand if you can