Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .envrc
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is there purely for ergonomics of people who are using direnv-nix.

"use flake" directive means "dear direnv, when the user enters this directory, find file flake.nix and evaluate it with nix develop".

What happens when you run nix develop is that nix installs all the necessary packages into /nix/store behind their input hashes and then "activates" a mutually-compatible ser of packages, giving access to them in PATH.

Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
use flake

3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,6 @@ pcre_constants.include
junk

.vagrant/

.direnv/
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Direnv creates .direnv to maintain its state.

result
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nix creates result/ directory when you run nix build. Symlinks to built artefacts in /nix/store live there

61 changes: 61 additions & 0 deletions flake.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

75 changes: 75 additions & 0 deletions flake.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
{
description = "NGS - Next Generation Shell";
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the name of the show!


inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks scarier than it is. All the cool kids are on unstable.

I used to use Arch, btw

flake-utils.url = "github:numtide/flake-utils";
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nix build system didn't "find" flake framework of unification of inputs / outputs for Nix expression straight away. Before we used to write simple nix programs which would be [lazily] evaluated top to bottom.

flake-utils is like flake framework stdlib.

It has some convenience functions like generating an attrset per each system supported by default (used here), to flatten a bunch of nested attributes into a flat attrset , etc.

};

outputs =
{
self,
nixpkgs,
flake-utils,
}:
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a function saying "give me a reference to the outputs of this function (recursive), everything that is packaged in whatever the source of nix packages is (unstable "channel" of official nix packages in this case) and flake utils, and I will give you an attrset defined below".

flake-utils.lib.eachDefaultSystem (
system:
let
pkgs = nixpkgs.legacyPackages.${system};

# Local source filter - exclude build artifacts
localSrc = builtins.path {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Source code is very important to build software!

We only care about the actual source tree, not build artefacts or git object store.

path = ./.;
name = "ngs-source";
filter =
path: type:
let
baseName = baseNameOf path;
in
!(baseName == "build" || baseName == "result" || baseName == ".git" || baseName == ".direnv");
};

# For local development, override the package to use local source
ngs = (pkgs.callPackage ./package.nix { }).overrideAttrs (oldAttrs: {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we buuld ngs for nixpkgs (this is my end goal with this work -- to have ngs included into official nixpkgs unstable), we want to pin the version to a certain place fetchable with some fetcher. We use github fetcher, because github fetcher is so easy to use.

In package.nix you will see that src is remote!

But to build stuff locally, we need to override this source with local source.

src = localSrc;
version = oldAttrs.version + "-dev";
});
in
{
packages = {
default = ngs;
ngs = ngs;
};

devShells.default = pkgs.mkShell {
inputsFrom = [ ngs ];
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is where the package is coming from, so we just reuse release dependencies from there. We separate concerns and oly add developmnent dependencies here like valgrind profiler, gdb, and other tools I'm scared to launch.


packages =
with pkgs;
[
# Development tools
gdb
clang-tools # clangd, clang-format
]
++ lib.optionals stdenv.isLinux [
valgrind
strace
];

shellHook = ''
echo "╔══════════════════════════════════════════════════════════╗"
echo "║ NGS Development Environment ║"
echo "╠══════════════════════════════════════════════════════════╣"
echo "║ Build: cmake -B build && cmake --build build ║"
echo "║ Test: cd build && ctest --output-on-failure ║"
echo "║ Run: NGS_PATH=lib ./build/ngs ║"
echo "║ Package: nix build ║"
echo "╚══════════════════════════════════════════════════════════╝"
'';
};

# For nix fmt
formatter = pkgs.nixpkgs-fmt;
}
);
}
100 changes: 100 additions & 0 deletions package.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
{
lib,
stdenv,
fetchFromGitHub,
fetchurl,
cmake,
pkg-config,
pandoc,
gawk,
makeWrapper,
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I need to emphasise this one!

When CMake installs your software, it assumes LFS compliance.

With nix everything lives in /nix/store, thus at runtime software built with Nix often can't find libraries.

makeWrapper replaces global runtime assumptions with explicit, reproducible runtime configuration.

But one picture is better than 1000 words:


sweater in pentavus~/Github/ngs on  master via C v21.1.2-clang via △ v4.1.2 via 𝗩 via  impure (nix-shell-env) on  (us-east-1) 
λ nix build

sweater in pentavus~/Github/ngs on  master via C v21.1.2-clang via △ v4.1.2 via 𝗩 via  impure (nix-shell-env) on  (us-east-1) 
λ cat result/bin/ngs
#! /nix/store/p0k9r5h8qs7220xdbdihhfgzwjcly70x-bash-5.3p3/bin/bash -e
export NGS_PATH='/nix/store/lhlnv03mjklkc8dwc2gvmxs6axwshbzg-ngs-0.2.17-dev/lib/ngs'
exec -a "$0" "/nix/store/lhlnv03mjklkc8dwc2gvmxs6axwshbzg-ngs-0.2.17-dev/bin/.ngs-wrapped"  "$@"

sweater in pentavus~/Github/ngs on  master via C v21.1.2-clang via △ v4.1.2 via 𝗩 via  impure (nix-shell-env) on  (us-east-1)
λ head -n1 result/bin/.ngs-wrapped
����
    ��� �H__PAGEZERO��__TEXT@@__text__TEXTD0(uD0�__stubs__TEXTl��l�
                                                                   __const__TEXT �T �__cstring__TEXTt���t�__unwind_info__TEXT$9�$9��__DATA_CONST@@@@�__got__DATA_CONST@�@���__DATA����__data__DATA��S�__common__DATA��

boehmgc,
json_c,
libffi,
pcre,
# Darwin-specific: GNU sed is required for build scripts
gnused,
}:
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is understandable, it just follows the list of dependencies that you have listed in your scripts.


stdenv.mkDerivation (finalAttrs: {
pname = "ngs";
version = "0.2.17";
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apr 5th release


src = fetchFromGitHub {
owner = "ngs-lang";
repo = "ngs";
rev = "v${finalAttrs.version}";
hash = "sha256-j7OAXHADc2LlabKxVgYiKeDDtLttDVIavhQZSGyPGlE=";
};

# NGS requires peg 0.1.18 specifically due to custom patches in build-scripts/
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was a bit of a pain in the rear to package, but I managed.

I'm lucky that it seems that everything is mutually compatible between nixpkgs-unstable and peg-0.1.18!

# The build scripts patch the leg output for location tracking, and this
# patching is not compatible with newer peg versions (0.1.20+).
# CMake will download and build peg 0.1.18 automatically when leg is not found.
pegSrc = fetchurl {
url = "https://www.piumarta.com/software/peg/peg-0.1.18.tar.gz";
hash = "sha256-IBk73Wc/x0h6OJN+KX//CKpzdRtjOghqwow7NIkPkIQ=";
};

nativeBuildInputs =
[
cmake
pkg-config
pandoc
gawk
makeWrapper
]
++ lib.optionals stdenv.isDarwin [
gnused
];

buildInputs = [
boehmgc
json_c
libffi
pcre
];

# The build scripts require GNU sed; on Darwin we need to ensure it's found
# Also pre-populate the peg source to avoid network access during build
preConfigure =
''
# Create the external project download directory
mkdir -p build/leg-prefix/src
# Copy peg source to where CMake ExternalProject expects it
cp ${finalAttrs.pegSrc} build/leg-prefix/src/peg-0.1.18.tar.gz
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we are tricking CMake into thinking that it already downloaded the peg code by placing it where it expects.

The reason we have to maintain our patches ourselves is that nix is hermetic and side-effect-free at the build time, meaning it can't just download stuff from the internet willy nilly.

''
+ lib.optionalString stdenv.isDarwin ''
export PATH="${gnused}/bin:$PATH"
'';

cmakeFlags = [
"-DBUILD_MAN=ON"
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pure cargo cult

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(as in, I didn't check that it actually builds manual)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mmm. We don't expect people to man ngs?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, what I mean is that I just copied this flag without checking it. I don't even know if it's real. I do expect people to man ngs, I plan to even do it myself. Just flagging that I didn't check that it actually works, I just did this:


$ ls result/share/man/man1/
na.1.gz  ngs.1.gz  ngsint.1.gz  ngslang.1.gz  ngsstyle.1.gz  ngstut.1.gz  ngswhy.1.gz

but I don't know if the man pages are generated because of this flag and I don't know if they are populated correctly. :)

];

# Set NGS_PATH so the installed binary can find the standard library
postInstall = ''
wrapProgram $out/bin/ngs \
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrapProgram is coming from makeWrapper package and this is what makes the wrapper that I showed you before.

sweater in pentavus~/Github/ngs on  master via C v21.1.2-clang via △ v4.1.2 via 𝗩 via  impure (nix-shell-env) on  (us-east-1)
λ cat result/bin/ngs
#! /nix/store/p0k9r5h8qs7220xdbdihhfgzwjcly70x-bash-5.3p3/bin/bash -e
export NGS_PATH='/nix/store/lhlnv03mjklkc8dwc2gvmxs6axwshbzg-ngs-0.2.17-dev/lib/ngs'
exec -a "$0" "/nix/store/lhlnv03mjklkc8dwc2gvmxs6axwshbzg-ngs-0.2.17-dev/bin/.ngs-wrapped"  "$@"

--set NGS_PATH $out/lib/ngs
'';

meta = with lib; {
description = "Next Generation Shell - a powerful programming language and shell designed for Ops";
longDescription = ''
NGS is a unique combination of select features borrowed from other
languages and original features. NGS was built from the ground up
focusing on daily systems engineering tasks.
One way to think about NGS is bash plus data structures plus better
syntax and error handling. Scripting AWS is much easier with NGS,
there is a Declarative Primitives style library for that.
'';
homepage = "https://ngs-lang.org/";
changelog = "https://github.com/ngs-lang/ngs/blob/v${finalAttrs.version}/CHANGELOG.md";
license = licenses.gpl3Only;
maintainers = with maintainers; [ ];
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do I do when something breaks and I don't have enough understanding how to fix this?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm afraid I will have to be maintainer then, the issue is that I'm currently using YSH and I wanted to evaluate NGS as a better alternative. There is also a potential that someone in a huge nix community wants to take up maintenance of this expression.

We will see!

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you like the idea largely. I will put myself as a maintainer then and will remove myself if I feel like I can't do maintenance of it anymore or when I find someone else trustworthy to maintain this set of exprs.

mainProgram = "ngs";
platforms = platforms.unix;
};
})
Loading