Editor: Introduce FileImporter

[ycw]

About this PR

In order to make file-importing maintainable and readable, I created FileImporter, and bunch of file handler classes(in file/handlers/) instead of using single giant file Loader

The end goals

• Flattens the import flow in each file handler; easier to track async bugs:

async import( file ) {

	const loader = await this.instantiate( 'three/addons/loaders/ColladaLoader.js' );

	const contents = await readAsText( file );

	const object = loader.parse( contents );

	object.scene.name = file.name;

	this.editor.execute( new AddObjectCommand( this.editor, object.scene ) );

}

• Easier to trap parsing error in each file handler for better UI Feedback, help distinguish from unsupported file format error: (not included in this PR)

async import( file ) {

	const loader = await this.instantiate( 'three/addons/loaders/ColladaLoader.js' );

	const contents = await readAsText( file );
// try
	const object = loader.parse( contents );
// catch
	object.scene.name = file.name;

	this.editor.execute( new AddObjectCommand( this.editor, object.scene ) );

}

• Easier to extend, say for 'export' in some file formats: (not included in this PR):

class FileHandler_OBJ extends FileHandler {
  async import( file ) { .. } 
  async export( object ) { .. }  // <<<
} 

• File-related utils all goes to single place FileUtils, easier to maintain:

  • FileUtils.formatFileSize instead of editor.utils.formatNumber
  • FileUtils.getFileExt instead of implementing it sparsely in Loader
  • FileUtils.readAsText a wrapper of reader, returning promise, help flattens the import flow(see above code snippet)
  • ...

• Adds introspection of supported file extensions, i.e. no hardcoded list, easier to maintain:

  • FileImporter.getAcceptedFileExt ... e.g. generate hint in UI Feedback when users inputted unsupported file formats

• Adds FileManager, an abstraction of files cache, can normalize all file-import paths by building an artificial FileManager from FileList(File>Import), DataTransferItemList(Drag n Drop), or unzipped File[](when user inputted a ZIP via File>Import or DnD), easier to maintain.

• Each file handle in standalone class, easier to read and maintain, in particular when searching symbols, fewer irrelevant matches.

---

QA

Why introduce new class instead of improving Loader

The name 'FileImoprter' is chosen because it has better semantic, i.e. files will be imported to the scene, instead of 'Loader' which only loads file into memory and not add to the scene.

Why file handler in (file/handlers) has FileHandler_XXX name pattern

Because some file extensions start with a number, for example .3ds, so instead of aliasing like ThreeDSFileHandler, simply putting the ext name at the tail, FileHandler_3DS.

Why only one `import()` method in FileImporter

In Loader, there're 3 methods, one for loading single file, one for loading multiple files, one for loading DataTransferItemList.
However, there's no use cases of loading single file: file input .files is FileList, drag n drop yields DataTransferItemList, so FileImporter has only one import method import( files ) which covers File[], FileList, and DataTransferItemList.

Why file handler is a class not a function

• More intuitive comparing to function interface, as you can see using functions required passing many arguments, some is just stub values in order to satisfy the fn interface. In contrast, FileImporter will delegate file-import to specific handler, e.g. call FileHandler_GLTF.import(file) to handle a gltf file, i.e. just the file is passing around, no stub values.

• Easier to extend for the future, for example, file handler FileHandler_GLTF may implements export(object), and then FileExporter delegates file-export of gltf to it.

Whats the value of .instantiate() in FileHandler (base class)

.instantiate() is simple and is 'great to have' because:

1. It simplified the two-step import Loader + new Loader.
2. It ensures that the FileHandler's FileManager must be propagated to the loader

.instantiate() plus other FileUtils creates a flat import() flow, it helps debugging, in future PR, try/catch is made to trap parsing error and showing proper UI feedback:

async import( file ) {

	const loader = await this.instantiate( 'three/addons/loaders/ColladaLoader.js' );

	const contents = await readAsText( file );
//try
	const object = loader.parse( contents );
//catch
	object.scene.name = file.name;

	this.editor.execute( new AddObjectCommand( this.editor, object.scene ) );

}

What is FileManager

FileManager is just a LoadingManager, it abstracted "files cache", When calling FileImporter.import( files ), a FileManager can be provided in the 2nd parameter, in this case, FileImporter will try 'loading files from the "file cache"' before trying fetch files from the internet.

This mechanics normalized different ways of file-import, via creates a artificial FileManager for loaded files, while these files may come from file input, drag n drop, or even extracted from a ZIP:

• File input (menubar.file.js) -> editor.fileImporter.import( files ) (changes included in this PR)
• Drag n drop (index.html) -> editor.fileImporter.import( files ) (changes included in this PR)
• UnZIP (FileHandler_ZIP.js) -> fileImporter().import( files ) (changes included in this PR)

i.e. With FileManager, users can import a ZIP of any supported file formats, as long as the corresponding loaders honors loadingManager.

Alternatives of Loader.getSupportedFileFormat

Accepted file format is now FileImporter.getAcceptedFileExts(), extension includes dot e.g. '.md2' not 'md2', so Menubar.file.js needs a subtle change (included in this PR)

Details

LoaderUtils

(fn) createFilesMap: no needed, implemented in FileManager

(fn) getFilesFromItemList: moved to FileUtils (now returns Promise)

Loader.js

(utils) handleJSON: moved to FileHandler_JS

(utils) handleZIP: no needed, rewrote in FileHandler_ZIP

(utils) createGLTFLoader: moved to FileHandler_GLTF

(static) getSupportedFileFormats: no needed, uses FileImporter.getAcceptedFileExts() instead

(method) loadItemList: no needed, uses new FileImporter().import() instead (covered DataTransferItemList)

(method) loadFiles: no needed, uses new FileImporter().import( files ) instead

(method) loadFile: no needed, uses new FileImporter().import( [ file ] ) instead

FileImporter and FileManager

file/FileManager

• is-a LoadingManager; act as files cache store, an abstraction of

  • files selected via file input
  • files via drag n drop
  • files extracted from ZIP

file/FileImporter

• import(..): imports files to editor (delegates to specific handler)

• canImport(..): check if any handler can handle the file

• getAcceptedFileExts(): arr of accepted file exts, ext incl dot, e.g '.md2'

FileHandler, FileHandler_XXX and FileHandlers

file/handlers/FileHandler:

• base class; with a handy instantiate() simplified import + new

file/handlers/FileHandler_XXX:

• derived classes; implements import(file) returning promise.

file/handlers/FileHandlers:

• index class; with a handy getImportHandler(..) to get proper handler for given file

Previews

Preview: https://raw.githack.com/ycw/three.js/editor-new-import-export-system/editor/index.html

Code: https://github.com/ycw/three.js/tree/editor-new-import-export-system/editor/js/file

#28324 (comment) Solved.

[mrdoob]

I'm sorry... But I think 1 file is easier to maintain than 32 files... 😔

[mrdoob]

Sorry, but I think you're over engineering this.

[ycw]

Sorry, but I think you're over engineering this.

Updated for clarification, would you mind review again?