HOCON vs JSON decision

[HelloAnneKim]

This isn't so much an issue but more of a point of clarification. Why was HOCON chosen over JSON. I saw the decision briefly described as "cleaner" than JSON but that seems kind of subjective. Objectively, JSON is more familiar and introduces fewer potential problems of incompatibility. I recently tried to implement a Flower + FLARE architecture that became a bit more complicated by the HOCON convention in FLARE.

[chesterxgchen]

HOCON is mainly for human readable json format. Although "cleaner" or not can be subjective. There are some solid benefits.

Just add the context why we support HOCON and YAML in additional to JSON. Before FLARE add python API, the configuration file is essentially the interface for data scientist to configure the components ( controller, executor, filters and other plugin components). Json works fine if you understand what you are editing.

Since JSON doesn't support comment, its hard to explain to the users what the configuration is for directly in the JSON file, we have to use README or documentation to do it. On top of that, we also like to be able to not repeat the a lot of the configuration files to allow inheritance, include and many other features that are not available in JSON, but readily available in HOCON and YAML file formats.

HOCON adding additional features which maybe nice for some, and other may not care. for example instead using
"A": "B", you can simple A = B

With HOCON, it a lot of easier for us to build JOB template, directly add comment at different lines and explain what is suppose to be etc.

But of course, the importance of this feature become decreased with our introduction of Python APIs, where the JSON configuration can be dynamically generated with a few lines of python code.

This might be still important for some system integrators where the configuration editing and manipulation is important.

Hope this give you some perspective