Issues with persistency in storage File/S3/… with setting engine_file_allow_create_multiple_files

[CurtizJ]

Describe the unexpected behaviour
All files created by storage with setting engine_file_allow_create_multiple_files except the first one are not visible by the storage after restart.

How to reproduce

DROP TABLE IF EXISTS test_file_insert;

CREATE TABLE test_file_insert (x UInt64) ENGINE = File(JSON, 'test_file.json');

SET engine_file_allow_create_multiple_files = 1;

INSERT INTO test_file_insert SELECT * FROM numbers(10);

INSERT INTO test_file_insert SELECT * FROM numbers(10);

SELECT _file, count() FROM test_file_insert GROUP BY _file ORDER BY _file ASC;

   ┌─_file────────────┬─count()─┐
1. │ test_file.1.json │      10 │
2. │ test_file.json   │      10 │
   └──────────────────┴─────────┘

DETACH TABLE test_file_insert;
ATTACH TABLE test_file_insert;

SELECT _file, count() FROM test_file_insert GROUP BY _file ORDER BY _file ASC;

   ┌─_file──────────┬─count()─┐
1. │ test_file.json │      10 │
   └────────────────┴─────────┘

Expected behavior
The persistent storage with engine File should read from the same set of files after restart.