Making a multi-phased extensions #12

mkikets99 · 2022-05-15T06:38:30Z

This also includes:

cosmetic changes
bug fixes
preparation for iso build
folder-like static and templates

Mkikets99 dev

Preparation for "Multi_Phase" Extensions

Also: prepared for iso build, added --phase-id option(optional), made output cosmetics, moved config(CHROOT part) to separate file(will be changed in next commit

mkikets99 · 2022-05-15T07:11:21Z

I am going to change a way how it execute phases.
It will be a loop with executing of same command strokes:

run_before_{PHASENAME} function will be skipped if not exists
build_phase {PHASENAME} {SECTION} function
run_after_{PHASENAME} function will be skipped if not exists
$ONEOFF && $FOUNDSTARTPHASE && unmount_image && exit

Is it ok?

Added `run_before` and `run_after` functions also done folder like style for `static` and `templates`

krglaws · 2022-05-15T14:13:29Z

version-check.sh

+if [ -x dummy ]
+  then echo "g++ compilation OK";
+  else echo "g++ compilation failed"; fi
+rm -f dummy.c dummy


This file isn't necessary since it's already built in to the script with mylfs.sh --check

Yes, but there is some output issues with some files, like yacc...
I think that ill just set it in .gitignore...

krglaws · 2022-05-15T14:14:19Z

mylfs.sh

-# unmount and detatch image
-unmount_image
-
-echo "build successful."


I like that you moved this into a separate function

I moved it cause iso will work separately from building

yeah makes complete sense, even without the iso build

krglaws · 2022-05-15T14:16:06Z

README.md


 ```
-$ sudo ./build.sh --help
+$ sudo ./mylfs.sh --help


oof, can't believe I missed that

That what i meant in #6

krglaws · 2022-05-15T14:19:11Z

lfs/config.sh

+    rm -rf $LFS/usr/share/{info,man,doc}/*
+    find $LFS/usr/{lib,libexec} -name \*.la -delete
+    rm -rf $LFS/tools
+}


Interesting, I like this

this will make Sections with phases more mobile and modular

krglaws · 2022-05-15T14:31:54Z

pkgs.sh

 export PATCH_GLIBC=https://www.linuxfromscratch.org/patches/lfs/11.1/glibc-2.35-fhs-1.patch
 export PATCH_KBD=https://www.linuxfromscratch.org/patches/lfs/11.1/kbd-2.4.0-backspace-1.patch
 export PATCH_PERL=https://www.linuxfromscratch.org/patches/lfs/11.1/perl-5.34.0-upstream_fixes-1.patch
+export PATCH_SYSLINUX=https://gitweb.gentoo.org/repo/gentoo.git/plain/sys-boot/syslinux/files/syslinux-6.03-sysmacros.patch


So I understand syslinux is some kind of bootloader. Will it "replace" GRUB whenever the script builds an ISO? Or will they need to coexist?

Grub will operate a boot of vmz and imglinux files. Syslinux helps to boot them from fat systems on which iso operates and boots.

This means that they will cooperate...
But RN i have some issues with building syslinux... Must check how it built from that project.

krglaws · 2022-05-15T14:55:40Z

README.md

+
+        -t|--phase-id       Specify which portion of script you need to execute. If 
+                            not specified, will run 'lfs' scripts.
+


Correct me if I'm wrong, but it seems like we would only be able to build one "STARTSECTION" at a time. Is that right? I kind of like the idea of being able to build everything I need in a single command, like so:

sudo ./mylfs.sh --build-all --extend path/to/extension
rather than
sudo ./mylfs.sh --build-all --phase-id lfs && sudo ./mylfs.sh --build-all --phase-id path/to/extension

And I kind of prefer the --extend parameter name to the --phase-id parameter name. But I like where you are going with grouping phases together under a single directory.

so my_extension/ might have:
phase1/
phase2/

sudo ./mylfs.sh --build-all --extend path/to/my_extension would build the lfs phases AND the my_extension phases all in one go. Thoughts?

Hmm... what if make --extend as array? So when you add more of them they will build up in a list and build one by one...

Then i will just edit some of Build Extension function to work with multiple values
And it will auto start after the building...
But then something additional option needed, cause sometimes it can broke on extension, but not on the lfs part

Yeah, storing it as an array could work. And you could pass the extensions in either like --extend path1 --extend path2... or --extend path1,path2,.... Up to you.

As for restarting after a break in an extension, you could add another param like you mentioned:

# after a failure in 'my_extension' sudo ./mylfs.sh --extend my_extension --start-extension my_extension --start-phase 2 --start-package some_package

We should also make it clear that the order in which you specify extensions is important, since that will be the order in which they are built.

ok. so i'll redo changes for --phase-id and extend more functionality on --extend

And you could pass the extensions in either like --extend path1 --extend path2... or --extend path1,path2,.... Up to you.

I am more prefer to have a multiple, or separated with : - like a PATH separation
So... About multiple - it will be working soon.
A oneliner - must do a separation function. But after that it will work with any symbol.

mkikets99 · 2022-05-15T15:09:52Z

mylfs.sh

+    HOUT+=$(repeat_echo ${#SSTR} '#';echo)
+    HOUT+="\n# ${SSTR}\n"
+    HOUT+="# $(repeat_echo ${#SSTR} '~';echo)"
+    echo -e $HOUT


Also: this is a cosmetic change
This will make automatic top and bottom line for text

krglaws · 2022-05-15T15:39:30Z

regarding the missing GRUB build, it seems that one of the build scripts is messing with STDIN (which affects the while read loop), and that's why it skips the grub line in build_order.txt. https://stackoverflow.com/questions/13800225/while-loop-stops-reading-after-the-first-line-in-bash

I'm thinking one of the commands in phase4/linux.sh must be reading STDIN and accidentally pulling "grub" out of the stream before the while loop gets to it.

Possible fixes:

manually run the linux script in chroot to find out which command is reading stdin
change the while loop in build_phase to not rely on stdin, and thereby keep the loop safe from whatever the scripts are doing

mkikets99 · 2022-05-15T16:27:32Z

regarding the missing GRUB build, it seems that one of the build scripts is messing with STDIN (which affects the while read loop), and that's why it skips the grub line in build_order.txt. https://stackoverflow.com/questions/13800225/while-loop-stops-reading-after-the-first-line-in-bash

I'm thinking one of the commands in phase4/linux.sh must be reading STDIN and accidentally pulling "grub" out of the stream before the while loop gets to it.

Possible fixes:

manually run the linux script in chroot to find out which command is reading stdin

change the while loop in build_phase to not rely on stdin, and thereby keep the loop safe from whatever the scripts are doing

I'm on my way to check it...
I'll search it on lfs book, and its rebuilding again with # on linux,grub and syslinux

There is also something with syslinux... it can't build cause it can't link something... more old script analyzing on the go

mkikets99 · 2022-05-15T17:35:53Z

regarding the missing GRUB build, it seems that one of the build scripts is messing with STDIN (which affects the while read loop), and that's why it skips the grub line in build_order.txt. https://stackoverflow.com/questions/13800225/while-loop-stops-reading-after-the-first-line-in-bash

I'm thinking one of the commands in phase4/linux.sh must be reading STDIN and accidentally pulling "grub" out of the stream before the while loop gets to it.

Possible fixes:

manually run the linux script in chroot to find out which command is reading stdin

change the while loop in build_phase to not rely on stdin, and thereby keep the loop safe from whatever the scripts are doing

Founded. it is on make command:

Enable legacy fbdev support for your modesetting driver (DRM_FBDEV_EMULATION) [Y/n/?] (NEW)
1.1. if Y - Overallocation of the fbdev buffer (DRM_FBDEV_OVERALLOC) [100] (NEW)
....
Enable firmware EDID (FIRMWARE_EDID) [N/y/?] (NEW) - FB section is on so it is called
Enable legacy fbcon hardware acceleration code (FRAMEBUFFER_CONSOLE_LEGACY_ACCELERATION)
.....
Bootup logo (LOGO) [N/y/?] (NEW)
Select compiled-in fonts (FONTS) [N/y/?] (NEW)
If you want i can check all Keys with their default values that are needed to add.

mkikets99 · 2022-05-15T18:17:41Z

@krglaws
Yoo-hoo.. having a progress with VM...
It loaded a Kernel-panic.... at least it loaded a vmz for it
I'll send some picks on discussion page

krglaws · 2022-05-15T19:49:00Z

Awesome! I'm gonna be AFK most of today and tomorrow probably. But I'll be able to check in here and there. Seems like we are going to need to fix those prompts in the linux.sh script. I'll think about it in the meantime

mkikets99 · 2022-05-16T07:48:44Z

One more edit on build_extension and i'll remove mainLFSbuild and set a build_extension lfs for it

Preparing to have a list of linux kernel parameters that are out of the list

Testing if everything working properly

Next: - Linux kernel build fixup - Searching for `vmz` aka `vmlinux` build in BLFS - Defining whats wrong with `syslinux` build - remaking `example_extension` folder to be like `lfs-build` folder

mkikets99 · 2022-05-17T09:55:13Z

The last one needs testing on that section...
Also, i must add checker if LFS built completely, or not... so extensions would not start if it is not...
Also, i think, it would be better if Build instructions for build_phase and build_package would be in config.sh or build.config.sh

- Moved syslinux to phase5 - `logs` are now inside of extensions, Or it is better to be `logs/$SECTION/$PKG.log`?

… while building phases (and clearing after it) and now no input issue on`read` command inside of `build_phase` Also some TODO.txt enhancements and echo additions (like `Ending of building phase`

thinking on a idea with pkgs_used.txt

…main # Conflicts: # mylfs.sh

…timer with `--time`, fixed issue with $NAMEOVERRIDE to be empty

mkikets99 added 13 commits May 12, 2022 22:02

Update binutils.sh, build_order.txt, and 220 more files...

645383f

phases in strings

4e99148

fixes to be on go with #e6a78ed

2e7a027

Merge branch 'main' into mkikets99_dev

1ea1fa3

Merge pull request #1 from mkikets99/mkikets99_dev

2bf219c

Mkikets99 dev

Revert with idea of BLFS and CLFS...

df72a7b

Preparation for "Multi_Phase" Extensions

reset

65252d0

moved to lfs dir

690fb4a

Changd the logic for have a multi phased extension (not ready yet)

6348073

Also: prepared for iso build, added --phase-id option(optional), made output cosmetics, moved config(CHROOT part) to separate file(will be changed in next commit

Merge branch 'krglaws:main' into main

176128c

Update TODO.txt and syslinux.sh

11d406f

Merge branch 'main' of https://github.com/mkikets99/MyLFS.git

b700867

renamed to be reliable, cosmetic changes and syslinux

f1e1aa1

mkikets99 added 2 commits May 15, 2022 15:38

moved templates and statics to Phase dir.

7dbd21c

Added `run_before` and `run_after` functions also done folder like style for `static` and `templates`

fixes for "Usage" part

1e8ef49

krglaws reviewed May 15, 2022

View reviewed changes

mkikets99 commented May 15, 2022

View reviewed changes

Update .gitignore, build_order.txt, and 3 more files...

e9f1d61

mkikets99 added 5 commits May 16, 2022 15:51

preparing Extensions, fixing Grub building...

98b9b53

Preparing to have a list of linux kernel parameters that are out of the list

Some comments for .gitignore

cbf7158

added counter to download, and made lfs build as "extension"

0f65ddf

Testing if everything working properly

Cleanning some mess and changing $SECTION to $EXTENSIONNAME

499e3bf

More clean up. Added functionality for -t|--start-section option.

01943f6

Next: - Linux kernel build fixup - Searching for `vmz` aka `vmlinux` build in BLFS - Defining whats wrong with `syslinux` build - remaking `example_extension` folder to be like `lfs-build` folder

mkikets99 added 14 commits May 17, 2022 14:22

Added qemu conversion: Can be runned as user, not just root

1e98c78

Changes

7202f95

- Moved syslinux to phase5 - `logs` are now inside of extensions, Or it is better to be `logs/$SECTION/$PKG.log`?

Fixed issue with some static functions, script is now copying sources…

033f402

… while building phases (and clearing after it) and now no input issue on`read` command inside of `build_phase` Also some TODO.txt enhancements and echo additions (like `Ending of building phase`

Fixed some issues when not building

a21f368

fixes on NAME_OVERRIDE

9d00889

thinking on a idea with pkgs_used.txt

Merge branch 'main' of https://github.com/krglaws/MyLFS into krglaws-…

5570645

…main # Conflicts: # mylfs.sh

Merge branch 'krglaws-main'

d874554

Update fstab

a897dea

Merge update

3c891bf

moved clean-up to function, created pkgs_used, done copying, added …

2d2cbf6

…timer with `--time`, fixed issue with $NAMEOVERRIDE to be empty

Added global timer

d40a7a9

removed unnecesary file and some newlines

903ee8c

reemoved version-check from repo

4523674

Merge branch 'krglaws:main' into main

eac13fc


		-t\|--phase-id Specify which portion of script you need to execute. If
		not specified, will run 'lfs' scripts.

Making a multi-phased extensions #12

Are you sure you want to change the base?

Making a multi-phased extensions #12

Uh oh!

Conversation

mkikets99 commented May 15, 2022

Uh oh!

mkikets99 commented May 15, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

krglaws commented May 15, 2022

Uh oh!

mkikets99 commented May 15, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mkikets99 commented May 15, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mkikets99 commented May 15, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

krglaws commented May 15, 2022

Uh oh!

mkikets99 commented May 16, 2022

Uh oh!

mkikets99 commented May 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mkikets99 commented May 15, 2022 •

edited

Loading

mkikets99 commented May 15, 2022 •

edited

Loading

mkikets99 commented May 15, 2022 •

edited

Loading

mkikets99 commented May 17, 2022 •

edited

Loading