Project layout and structure reorganizing

[Json-To-String]

Barring #5 , project should work out of the box. However, this is hard for someone who is handed this directory, and in the interest of reproducibility and following best practices I want to do some restructuring and cleaning.

Some points to focus on:

• Images directory is a bit bloated since it contains a mixture of raw and augmented images, post training/results figures, and random datasets from SQL queries. These should be decoupled in some fashion.

• I want someone to be able to do the usual python -m venv my_env + pip install -r requirements.txt workflow to get up and running. Ideally a new user should only need to run one python script and maybe change some hyperparameters in a config file. Unfortunately I was a little reliant on testing in a bunch of different notebooks, and then nbconverting perseusResNet50.ipynb to a .py script to run on the cluster.

• The old way I handled datasets is a nightmare, intelligently using class methods like convention recommends makes this much cleaner.

[nickvazz]

Making environments and installing quickly (if not on windows) is very quick with uv - highly recommend going that route!

[nickvazz]

Images directory is a bit bloated since it contains a mixture of raw and augmented images, post training/results figures, and random datasets from SQL queries. These should be decoupled in some fashion.

Do you instead want to leave steps to regenerate the dataset exactly into some directory known by the repo via an environment variable? Or maybe some google drive/s3 bucket with the data to download into a specific directory?

[nickvazz]

Unfortunately I was a little reliant on testing in a bunch of different notebooks, and then nbconverting perseusResNet50.ipynb to a .py script to run on the cluster.

This might be a good excuse to try marimo

[Json-To-String]

Making environments and installing quickly (if not on windows) is very quick with uv - highly recommend going that route!

I've heard a lot of good things about uv, I'll try to build up an environment with it since it seems pretty lightweight and fast

Images directory is a bit bloated since it contains a mixture of raw and augmented images, post training/results figures, and random datasets from SQL queries. These should be decoupled in some fashion.

Do you instead want to leave steps to regenerate the dataset exactly into some directory known by the repo via an environment variable? Or maybe some google drive/s3 bucket with the data to download into a specific directory?

Mmm, it currently regenerates exactly into a directory, I think I'm happy with something like that for now since its not too many images, I'm guessing any future students will be using a laptop (but really ideally something decently strong).

Unfortunately I was a little reliant on testing in a bunch of different notebooks, and then nbconverting perseusResNet50.ipynb to a .py script to run on the cluster.

This might be a good excuse to try marimo

This looks great, I'll give it a go