TOML config support for JBang

[wfouche]

I have been working with Python's PEP 723 inline script metadata format and have come to like it a lot. It is simply a TOML data structure encoded in comments. Adapted to JBang it would allow the JBang configuration for Java / Kotlin / Groovy to be specified as shown in the example below:

///usr/bin/env jbang "$0" "$@" ; exit $?

// ||| jbang
// requires-java = ">=21"
// dependencies = [
//   "org.springframework.boot:spring-boot-starter-web:3.4.4",
// ]
// [java]
//   runtime-options = "-server -Xms2g -Xmx2g -XX:+UseZGC -XX:+ZGenerational"
// |||

import static java.lang.System.*;

public class test {

    public static void main(String... args) {
        out.println("Hello World");
    }
}

What are the benefits of this approach?

0. Standardized configuration format (TOML)
1. Easier to parse the configuration data. The Toml block is extracted and then processed by a Toml library (org.tomlj:tomlj:1.1.1).
2. Extensible configuration format, new options can easily be added.
3. Additional (optional) support could even be provided to externalize the JBang configuration to a file which could then be included in the JBang scriipt. But this will also make it slighly harder to determine if a script should be recompiled (the source did not change, but the config did).

///usr/bin/env jbang "$0" "$@" ; exit $?

// ||| jbang
// jbang-config = "config.toml"
// |||

import static java.lang.System.*;

public class test {

    public static void main(String... args) {
        out.println("Hello World");
    }
}

4. By sharing a configuration file, several related scripts could use the same runtime options that are specified in one place.
5. Editing and syntax highlighting support of TOML configuration files via extensions for VSCode, Intellij, etc.

6. A --config option could be added to JBang to override the configuration of a JBang script to use a different config.toml file.
7. A --config option would also allow files that do not contain any JBang comments to be provided with configuration data at runtime (if required).

// Run with JBang as:  jbang --config config.toml run  test.java
import static java.lang.System.*;

public class test {

    public static void main(String... args) {
        out.println("Hello World");
    }
}

8. Migration support could be provided to rewrite a JBang script file and group the JBang //? config options into a single TOML config comment block or to write the config to a TOML file.
9. If a external config file is used for the JBang config options, then script processing time would be reduced because the script does not have to be scanned for information.

[wfouche]

Java code to parse a TOML-encoded configuration comment block for JBang scripts.

//DEPS org.tomlj:tomlj:1.1.1

import org.tomlj.TomlParseResult;

StringBuffer tomlText = new StringBuffer("");
{
    List<String> lines = Files.readAllLines(Paths.get(scriptFilename));
    boolean found = false;
    for (String line: lines) {
        if (line.startsWith("// ||| jbang")) {
            found = true;
        }
        else if (line.startsWith("// |||")) {
            found = false;
            break;
        } else if (line.startsWith("// ")) {
            if (found) {
                if (tomlText.length() == 0) {
                    tomlText.append(line.substring(3));
                } else {
                    tomlText.append("\n" + line.substring(3));
                }
            }
        }
    }
}
TomlParseResult tpr = Toml.parse(tomlText.toString());

[quintesse]

Personally I don't really find it an improvement in readability :-)

And it would definitely break all backward-compatibility.

[wfouche]

Personally I don't really find it an improvement in readability :-)

Opinions will definitely differ on this. :-)

Config data expressed in a well-known standardized format is easier to comprehend than an invented-on-the-spot alternative. Toml is also easier to process and is extensible which is a huge benefit.

GraalPython with its JBangIntegration.java extension adds //PIP and //PIP_DROP config options. The current configuration paradigm limits extensibility.

Config options are currently (can be) scattered throughout the script. With TOML support all the config information must grouped into the same text block (enhancing readability).

JBang 1.0 could potentially support this functionality.

"Where there's a will, there's a way!"

[quintesse]

I don't think changing the format is an option at this late a stage. JBang might not be 1.0 yet, but throwing away 4+ years of stability in that aspect doesn't seem like a thing our users would appreciate. And supporting both formats would only complicate things.

Also, the current paradigm is completely flexible, the current tags are not fixed and in fact there are several projects out there that add their own.

And currently the tags can be scattered throughout the script, but in actuality nobody does so. (I did suggest in the past to stop parsing tags once the first non-tag line is found, basically assuring the lines must always be kept together, but that suggestion wasn't followed up upon).

[wfouche]

@quintesse, (just to make it clear) we are having two intertwined discussions.

(1) What is the best way to represent configuration options for a tool such as JBang. This is a purely abstract and scientific discussion. If we can rewind the clock and the first few lines of code were to be written today for JBang, then given what we know currently, would we encode dependencies as //DEPS or use the TOML list/array format for dependencies, or pick something even better than TOML? It is nice to have access to a hierarchal namespace (as supported by TOML) which makes it easier to extend the data set that is represented, so I'm of the opinion that using TOML is a better solution than using the current JBang config notation. It is important to have this type of discussion and gain valuable insights. If not to be used in JBang, the knowledge gained might be of use on another project.

(2) This is the where we face reality and weigh our options and in the context of JBang and its development history and user base, does it make any sense to change the way config options are represented? Maybe the ship has sailed on this, and it is just too late. Maybe not. Java 1.0 supported applets and it was an important use case. Applets are no longer supported in the Java language we use today. Software systems change and evolve over time. So will JBang change and evolve in the many years ahead.

[wfouche]

Option [jbang].integrations=false was added to the GraalPy config. Seems more readable to me than //NOINTEGRATIONS.

#!/usr/bin/env python-jvm

# restclient_graalpy.py
#
# /// jbang
# requires-graalpy = "==24.2.1"
# requires-java = ">=21"
# dependencies = [
#   "org.springframework.boot:spring-boot-starter-web:3.4.4"
# ]
# [graalpy]
#   allowAllAccess = true
#   emulateJython = true
# [java]
#   runtime-options = "-XX:+UnlockExperimentalVMOptions -XX:+EnableJVMCI -Dpolyglot.engine.WarnInterpreterOnly=false"
# [jbang]
#   integrations = false
# ///

import java.lang.String as String
import org.springframework.web.client.RestClient as RestClient

def restApiCall(uri: str, id: int):
    restClient = RestClient.create()
    rsp = restClient.get().uri(uri, id).retrieve().body(String)
    print(rsp)

def main():
    restApiCall("https://jsonplaceholder.typicode.com/todos/{id}", 1)

main()

[quintesse]

We tend to create flags whose default value is "false" and can be set to "true" to enable an "exceptional/non-default" situation.
That's why it's called "no integrations", because by default integrations are enabled and disabling them is "exceptional".