Feature Suggestion: Tupfile.base

[ppannuto]

I've found it a minor weakness / annoyance of tup that the first time I check out a copy of a repo, I have to remember to run tup init before I can tup upd. There's a few challenges with running init automatically, the most obvious of which is knowing where the root of the project directory is (assuming it's the current directory feels risky).

My proposal is the addition of a new file: Tupfile.base (name flexible, maybe Tupefile.root, Tupfile.ini?)

This file would be placed in the root directory of the project to identify it. In addition to identifying the root, I would propose a simple syntax (currently thinking .ini-style syntax) for expressing things such as default variants for the project.

In my current mental model, this file is parsed if and only if tup cannot find a .tup directory. More specifically, as tup upd begins enumerating directories searching for the .tup directory, it also searches for Tupfile.base (after looking for .tup). If a Tupfile.base is found, the requisite commands are run to initialize things and then tup continues on with upd.

I'm willing to tackle implementing this feature, but I'd like some input on the design first. Is this a good / desirable idea? Do people have comments / suggestions for improving it?

[bradjc]

I try to encourage people to use tup and adding a new step other than just typing make always seems like going backwards to them. Therefore I support something that will just run tup init automatically the first time.

My suggestion is to not only add support for a new file (I like Tupfile.root) but also for a new command, say root that you could add to a Tupfile in the root directory (if it exists) that would tell tup to run tup init in that folder. Maybe the parsing required to do this would be unreasonable, but if not it would cut down on an extra file in probably a lot of cases. The rest of the line after root could specify options somehow, maybe:

root variants=debug,release logging=true <whatever other options>

Just a thought.

[gittup]

On Wed, Jul 31, 2013 at 1:37 PM, Pat Pannuto notifications@github.comwrote:

I've found it a minor weakness / annoyance of tup that the first time I
check out a copy of a repo, I have to remember to run tup init before I
can tup upd. There's a few challenges with running init automatically,
the most obvious of which is knowing where the root of the project
directory is (assuming it's the current directory feels risky).

My proposal is the addition of a new file: Tupfile.base (name flexible,
maybe Tupefile.root, Tupfile.ini?)

This file would be placed in the root directory of the project to identify
it. In addition to identifying the root, I would propose a simple syntax
(currently thinking .ini-style syntax) for expressing things such as
default variants for the project.

In my current mental model, this file is parsed if and only if tup cannot
find a .tup directory. More specifically, as tup upd begins enumerating
directories searching for the .tup directory, it also searches for
Tupfile.base (after looking for .tup). If a Tupfile.base is found, the
requisite commands are run to initialize things and then tup continues on
with upd.

I'm willing to tackle implementing this feature, but I'd like some input
on the design first. Is this a good / desirable idea? Do people have
comments / suggestions for improving it?

This sounds good to me. I don't have any preference on the name - if it's
ini-style syntax, I guess .ini would be preferred for syntax-highlighting
reasons (tup already has the inih library builtin).

One question I have is what would the behavior be if multiple Tupfile.ini
files are found up the tree? I think in this case it might make sense to
pick the highest one (so, the last one found during the "up-the-tree"
traversal), so that if a sub-project has a Tupfile.ini, and a master
project also has a Tupfile.ini, the master project would be initialized.

ie:

super/Tupfile.ini
super/sub/Tupfile.ini

If there is no .tup directory, I think we want to make it at super/.tup

-Mike

[gittup]

On Wed, Jul 31, 2013 at 7:22 PM, Brad Campbell notifications@github.comwrote:

I try to encourage people to use tup and adding a new step other than just
typing make always seems like going backwards to them. Therefore I
support something that will just run tup init automatically the first
time.

My suggestion is to not only add support for a new file (I like
Tupfile.root) but also for a new command, say root that you could add to
a Tupfile in the root directory (if it exists) that would tell tup to run
tup init in that folder. Maybe the parsing required to do this would be
unreasonable, but if not it would cut down on an extra file in probably a
lot of cases. The rest of the line after root could specify options
somehow, maybe:

root variants=debug,release logging=true

Just a thought.

I think this would be difficult to add to the current Tupfile parser,
unless it is parsed with a separate parser that ignores all of the rule &
variable definitions normally in a Tupfile (otherwise, tup will try to
process dependent Tupfiles & such). I'd favor a separate file for this
purpose.

-Mike

[kurige]

I don't understand why this is so problematic that it requires adding a new file? You've already got '.tup', 'Tupfile', and 'Tuprules.tup' at the root directory. Adding a fourth 'tup' related item to a project's root is a little excessive, don't you think?

[ppannuto]

No, in this case you do not have .tup yet, that is the whole point of
this feature / file, to obviate tup init as a separate command. To do
that, you need some way of identifying the root directory of the project;
this is the purpose of the new file.
On Jul 31, 2013 6:38 PM, "Christopher Gateley" notifications@github.com
wrote:

I don't understand why this is so problematic that it requires adding a
new file? You've already got '.tup', 'Tupfile', and 'Tuprules.tup' at the
root directory. Adding a fourth 'tup' related file to a project's root is a
little excessive, don't you think?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/132#issuecomment-21908785
.

[ppannuto]

My current plan for implementation is to build on the existing option mechanism. Specifically, to create a dynamic array of locations adding a reference for each Tupfile.ini encountered from the directory where tup was invoked down to {the filesystem root, the .tup directory}, whichever is found first. Configuration lookup precedence would then first search this array and then fall back to the existing files.

As these are run-time configurations and orthogonal to the build, I believe they should not be tracked in tup's database (this also alleviates the chicken-egg problem for the first run). As these options potentially change the behavior of tup when executing, I believe the appropriate place to hook this scan is tup/main.c::127 (just before the comment /* Commands that don't use a normal tup_init() */.

As for nested .ini files, my solution would be to have a configuration "root" variable that defaults to false. When parsing the .ini's, if tup encounters root=true it will check the directory containing that .ini file for a .tup directory. If one is not found, it will execute tup init in that directory and stop traversing the directory tree.

This model alleviates some potential confusion for nested .ini's by making the marking of the root directory explicit and opens them to more general directory-sensitive configuration (which is really the only reason I could imagine nesting .ini files).

Thoughts?

[ppannuto]

It was brought to my attention that just the root=true idea would probably not be the best way of solving this. In particular, it would have weird / undesired effects on things like submodules in git, where you have nested projects, both of whom would define root=true.

The plans w.r.t. leveraging the options mechanism stand, but the new plan for directory search would be to go until a .tup directory is encountered or the root directory. If the search makes it all the way to the root, it will backtrack to the last Tupfile.ini it found. The expectation would be that this Tupfile.ini has root=true set and to use this as the project root. I think this is a good / flexible compromise.

As a final detail, if the shallowest .ini file didn't have root=true set, I think we'd have to emit a warning and ask the user what he'd like to do:

1. Run tup init in that directory anyway
2. Ignore that Tupfile.ini and go further back up the tree to the next one.

[gittup]

On Thu, Aug 1, 2013 at 6:30 PM, Pat Pannuto notifications@github.comwrote:

It was brought to my attention that just the root=true idea would
probably not be the best way of solving this. In particular, it would have
weird / undesired effects on things like submodules in git, where you have
nested projects, both of whom would define root=true.

The plans w.r.t. leveraging the options mechanism stand, but the new plan
for directory search would be to go until a .tup directory is encountered
or the root directory. If the search makes it all the way to the root, it
will backtrack to the last Tupfile.ini it found. The expectation would be
that this Tupfile.ini has root=true set and to use this as the project
root. I think this is a good / flexible compromise.

I don't see exactly why you'd need to use the options mechanism here. I was
thinking it would just do a stat("Tupfile.ini") on the way up the tree in
find_tup_dir(), and keep the topmost one (see attached patch).

As a final detail, if the shallowest .ini file didn't have root=true set,
I think we'd have to emit a warning and ask the user what he'd like to do:

1. Run tup init in that directory anyway
2. Ignore that Tupfile.ini and go further back up the tree to the next
   one.

Can you explain what the root=true is supposed to do? Why would I want a
Tupfile.ini that doesn't have root=true? My understanding is that a
Tupfile.ini file means that "this directory can house a .tup directory", so
they should all be "roots". To work properly with submodules, you would
want to init at the top-most root, which should be solved by just grabbing
the top-most Tupfile.ini as mentioned above.

Thanks!
-Mike

[bradjc]

I've talked to Pat about this and it seems that Tupfile.ini folds in nicely with the existing option infrastructure and option file hierarchy. Tupfile.ini would be added to the search order for tup options and would add repository specific options to the existing user specific options. This could potentially be useful for some projects to set, for instance, updater.full_deps to 1 if that particular project is expecting a lot of system changes.

Once there is a repo specific way to add options, setting the root location with root=true is a very explicit way to do so. Perhaps just using the Tupfile.ini closest to / makes more sense, however. It's possible that there might be folder specific Tupfile.ini files in subfolders in the future, but choosing the top Tupfile.ini would still work for finding the root. Maybe it's best to just not add support for root=true.

Also, making Tupfile.ini part of the options system allows for other repository specific options to be included with the repository. A very useful one would be default variants so that users don't have to run tup variant x.config and variant folders wouldn't have to be committed to the repository.

[ppannuto]

Brad covered the option point pretty well, as for the 'root' thing, you are
correct, it's not necessary if we choose the "lowest" initial file.

I have this mostly implemented, but will be traveling much of this week.
I'll push a branch out for review once I get a chance to finish things.
On Aug 4, 2013 3:58 PM, "Brad Campbell" notifications@github.com wrote:

I've talked to Pat about this and it seems that Tupfile.ini folds in
nicely with the existing option infrastructure and option file hierarchy.
Tupfile.ini would be added to the search order for tup options and would
add repository specific options to the existing user specific options. This
could potentially be useful for some projects to set, for instance,
updater.full_deps to 1 if that particular project is expecting a lot of
system changes.

Once there is a repo specific way to add options, setting the root
location with root=true is a very explicit way to do so. Perhaps just
using the Tupfile.ini closest to / makes more sense, however. It's
possible that there might be folder specific Tupfile.ini files in
subfolders in the future, but choosing the top Tupfile.ini would still work
for finding the root. Maybe it's best to just not add support for
root=true.

Also, making Tupfile.ini part of the options system allows for other
repository specific options to be included with the repository. A very
useful one would be default variants so that users don't have to run tup
variant x.config and variant folders wouldn't have to be committed to the
repository.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/132#issuecomment-22077746
.

[gittup]

I think we can close this, given the #133 has been merged.

[Qix-]

Why did we need an .ini file for this? I was always confused; whats wrong with just traversing upwards until we find a Tupfile, and if there isn't a .tup/ in the same directory tup init gets called? This seems like the correct behavior. Git does this, and nobody has ever had a problem with it.

[gittup]

That's a good idea, though you may not have a Tupfile at the root of your project if you don't build anything there.