From 3d475aea6b4148e3ef2467f14c2f4cc4b0755525 Mon Sep 17 00:00:00 2001 From: Alex <95913221+Pwhsky@users.noreply.github.com> Date: Mon, 10 Mar 2025 19:42:00 +0100 Subject: [PATCH 1/4] Update README.md --- README.md | 356 +++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 246 insertions(+), 110 deletions(-) diff --git a/README.md b/README.md index 8908cb7dd..1ad6bf944 100644 --- a/README.md +++ b/README.md @@ -1,176 +1,312 @@ +

- +

+ -

A comprehensive deep learning framework for digital microscopy.

+

DeepTrack2 - A comprehensive deep learning framework for digital microscopy.

- - PyPI version - PyPI version - Python version - - + + + + + PyPI version + + + + + + Python version + + +

Installation • - Examples • - Basics • + Getting Started • + Examples • + Advanced Tutorials • + Developer Tutorials • Cite us • License

+DeepTrack2 is a modular Python library for generating, manipulating, and analyzing image data pipelines for machine learning and experimental imaging. + +TensorFlow Compatibility Notice: +DeepTrack2 version 2.0 and subsequent do not support TensorFlow. If you need TensorFlow support, please install the legacy version 1.7. -We provide tools to create physical simulations of optical systems, to generate and train neural network models, and to analyze experimental data. +The following quick start guide is intended for complete beginners to understand how to use DeepTrack2, from installation to training your first model. Let's get started! # Installation -DeepTrack 2.1 requires at least python 3.6. +DeepTrack2 2.0 requires at least python 3.9. + +To install DeepTrack2, open a terminal or command prompt and run: +```bash +pip install deeptrack +``` +or +```bash +python -m pip install deeptrack +``` +This will automatically install the required dependencies. + +# Getting Started + +Here you find a series of notebooks that give you an overview of the core features of DeepTrack2 and how to use them: + +- DTGS101 **[Introduction to DeepTrack2](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/1-getting-started/DTGS101_intro.ipynb)** + + Overview of how to use DeepTrack 2. Creating images combining DeepTrack2 features, extracting properties, and using them to train a neural network. + +- DTGS111 **[Loading Image Files Using Sources](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/1-getting-started/DTGS111_datafiles.ipynb)** + + Using sources to load image files and to train a neural network. + +- DTGS121 **[Tracking a Point Particle with a CNN](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/1-getting-started/DTGS121_tracking_particle_cnn.ipynb)** + + Tracking a point particle with a convolutional neural network (CNN) using simulated images in the training process. + +- DTGS131 **[Tracking Multiple Particles with a U-Net](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/1-getting-started/DTGS131_tracking_multiple_particles_unet.ipynb)** + + Tracking multiple particles using a U-net trained on simulated images. + +- DTGS141 **[Distinguishing Particles with a U-Net](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/1-getting-started/DTGS141_distinguishing_particles_in_brightfield.ipynb)** + + Tracking and distinguishing particles of different sizes in brightfield microscopy using a U-net trained on simulated images. + +- DTGS151 **[Unsupervised Object Detection](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/1-getting-started/DTGS151_unsupervised_object_detection_with_lodestar.ipynb)** + + Single-shot unsupervised object detection a using LodeSTAR. + +# Examples + +These are examples of how DeepTrack2 can be used on real datasets: + +- DTEx211 **[MNIST](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/2-examples/DTEx201_MNIST.ipynb)** + + Training a fully connected neural network to identify handwritten digits using MNIST dataset. + +- DTEx212 **Single Particle Tracking** + + Tracks experimental videos of a single particle. (Requires opencv-python compiled with ffmpeg) + + +

+ + +
+ + + + + + +
+ + + + + + +

+ + +- DTEx213 **Multi-Particle tracking** +- + Detecting quantum dots in a low SNR image. + + +

+ +
+ + + + + + +
+ + + + + + +

+ + +- DTEx214 **Particle Feature Extraction** +- + Extracting the radius and refractive index of particles. + +- DTEx215 **Cell Counting** + + Counting the number of cells in fluorescence images. -To install DeepTrack 2.1, open a terminal or command prompt and run: +- DTEx216 **3D Multi-Particle tracking** - pip install deeptrack - -If you have a very recent version of python, you may need to install numpy _before_ DeepTrack. This is a known issue with scikit-image. + Tracking multiple particles in 3D for holography. -### Updating to 2.1 from 2.0 +- DTEx217 **GAN image generation** -If you are already using DeepTrack 2.0 (pypi version 0.x.x), updating to DeepTrack 2.1 (pypi version 1.x.x) is painless. If you have followed deprecation warnings, no change to your code is needed. There are two breaking changes: + Using a GAN to create cell image from masks. -- The deprecated operator `+` to chain features has been removed. It is now only possible using the `>>` operator. -- The deprecated operator `**` to duplicate a feature has been removed. It is now only possible using the `^` operator. +Specific examples for label-free particle tracking using **LodeSTAR**: -If you notice any other changes in behavior, please report it to us in the issues tab. +- DTEx231A **LodeSTAR Autotracker Template** -# Examples of applications using DeepTrack +- DTEx231B **LodeSTAR Detecting Particles of Various Shapes** -DeepTrack is a general purpose deep learning framework for microscopy, meaning you can use it for any task you like. Here, we show some common applications! +- DTEx231C **LodeSTAR Measuring the Mass of Particles in Holography** -
-

Single particle tracking

-

- - -
- Training a CNN-based single particle tracker using simulated data -
- - - - Unsupervised training of a single particle tracker using LodeSTAR +- DTEx231D **LodeSTAR Detecting the Cells in the BF-C2DT-HSC Dataset** + +- DTEx231E **LodeSTAR Detecting the Cells in the Fluo-C2DT-Huh7 Dataset** -

-
- -

Multi-particle tracking

- -

- -
- - - Training LodeSTAR to detect multiple cells from a single image -
- Training a UNet-based multi-particle tracker using simulated data -

-
+- DTEx231F **LodeSTAR Detecting the Cells in the PhC-C2DT-PSC Dataset** + +- DTEx231G **LodeSTAR Detecting Plankton** + +- DTEx231H **LodeSTAR Detecting in 3D Holography** -

Particle tracing

+- DTEx231I **LodeSTAR Measuring the Mass of Simulated Particles** + +- DTEx231J **LodeSTAR Measuring the Mass of Cells** -

- -
- - - Training MAGIK to trace migrating cells -

+Specific examples for graph-neural-network-based particle linking and trace characterization using **MAGIK**: + +- DTEx241A **MAGIK Tracing Migrating Cells** + + +

+ +
+ + + + + + +

+ + +- DTEx241B **MAGIK to Track HeLa Cells** + +# Advanced Tutorials + +This section provides a list of advanced topic tutorials. The primary focus of these tutorials is to demonstrate the functionalities of individual modules and how they work in relative isolation, helping to provide a better understanding of them and their roles in DeepTrack2. + +- DTAT301 **[deeptrack.features](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT301_features.ipynb)** + +- DTAT306 **[deeptrack.properties](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT306_properties.ipynb)** + +- DTAT311 **[deeptrack.image](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT311_image.ipynb)** + +- DTAT321 **[deeptrack.scatterers](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT321_scatterers.ipynb)** + +- DTAT323 **[deeptrack.optics](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT323_optics.ipynb)** -# Basics to learn DeepTrack 2.1 +- DTAT324 **[deeptrack.holography](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT324_holography.ipynb)** Open In Colab -Everybody learns in different ways! Depending on your preferences, and what you want to do with DeepTrack, you may want to check out one or more of these resources. +- DTAT325 **[deeptrack.aberrations](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT325_aberrations.ipynb)** -## Getting-started guides +- DTAT327 **[deeptrack.noises](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT327_noises.ipynb)** -We have two separate series of notebooks which aims to teach you all you need to know to use DeepTrack to its fullest. The first is a set of six notebooks with a focus on the application. +- DTAT329 **[deeptrack.augmentations](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT329_augmentations.ipynb)** -1. deeptrack_introduction_tutorial gives an overview of how to use DeepTrack 2.1. -2. tracking_particle_cnn_tutorial demonstrates how to track a point particle with a convolutional neural network (CNN). -3. tracking_particle_unet_tutorial demonstrates how to track multiple particles using a U-net. -4. characterizing_aberrations_tutorial demonstrates how to add and characterize aberrations of an optical device. -5. distinguishing_particles_in_brightfield_tutorial demonstrates how to use a U-net to track and distinguish particles of different sizes in brightfield microscopy. -6. analyzing_video_tutorial demonstrates how to create videos and how to train a neural network to analyze them. +- DTAT341 **[deeptrack.sequences](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT341_sequences.ipynb)** +- DTAT381 **[deeptrack.math](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT381_math.ipynb)** -The second series focuses on individual topics, introducing them in a natural order. +- DTAT383 **[deeptrack.utils](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT383_utils.ipynb)** -1. Introducing how to create simulation pipelines and train models. -2. Demonstrating data generators. -3. Demonstrating how to customize models using layer-blocks. +- DTAT385 **[deeptrack.statistics](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT385_statistics.ipynb)** Open In Colab -## DeepTrack 2.1 in action +- DTAT387 **[deeptrack.types](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT_387_types.ipynb)** Open In Colab -Additionally, we have seven more case studies which are less documented, but gives additional insight in how to use DeepTrack with real datasets +- DTAT389 **[deeptrack.elementwise](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT389_elementwise.ipynb)** Open In Colab -1. [MNIST](examples/paper-examples/1-MNIST.ipynb) classifies handwritted digits. -2. [single particle tracking](examples/paper-examples/2-single_particle_tracking.ipynb) tracks experimentally captured videos of a single particle. (Requires opencv-python compiled with ffmpeg to open and read a video.) -3. [single particle sizing](examples/paper-examples/3-particle_sizing.ipynb) extracts the radius and refractive index of particles. -4. [multi-particle tracking](examples/paper-examples/4-multi-molecule-tracking.ipynb) detects quantum dots in a low SNR image. -5. [3-dimensional tracking](examples/paper-examples/5-inline_holography_3d_tracking.ipynb) tracks particles in three dimensions. -6. [cell counting](examples/paper-examples/6-cell_counting.ipynb) counts the number of cells in fluorescence images. -7. [GAN image generation](examples/paper-examples/7-GAN_image_generation.ipynb) uses a GAN to create cell image from masks. +- DTAT391A **[deeptrack.sources.base](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT391A_sources.base.ipynb)** Open In Colab -## Model-specific examples +- DTAT391B **[deeptrack.sources.folder](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT391B_sources.folder.ipynb)** Open In Colab -We also have examples that are specific for certain models. This includes -- [*LodeSTAR*](examples/LodeSTAR) for label-free particle tracking. -- [*MAGIK*](examples/MAGIK) for graph-based particle linking and trace characterization. +- DTAT391C **[deeptrack.sources.rng](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT391C_sources.rng.ipynb)** Open In Colab -## Documentation -The detailed documentation of DeepTrack 2.1 is available at the following link: https://deeptrackai.github.io/DeepTrack2 +- DTAT393A **[deeptrack.pytorch.data](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT393A_pytorch.features.ipynb)** Open In Colab -## Video Tutorials +- DTAT393B **[deeptrack.pytorch.features](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT393B_pytorch.data.ipynb)** Open In Colab -Videos are currently being updated to match with the current version of DeepTrack. +- DTAT395 **[deeptrack.extras.radialcenter](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT395_extras.radialcenter.ipynb)** Open In Colab -## Cite us! -If you use DeepTrack 2.1 in your project, please cite us here: +- DTAT399A **[deeptrack.backend.core](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT399A_backend.core.ipynb)** Open In Colab + +- DTAT399B **[deeptrack.backend.pint_definition](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT399B_backend.pint_definition.ipynb)** Open In Colab + +- DTAT399C **[deeptrack.backend.units](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT399C_backend.units.ipynb)** Open In Colab + +- DTAT399D **[deeptrack.backend.polynomials](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT399D_backend.polynomials.ipynb)** Open In Colab + +- DTAT399E **[deeptrack.backend.mie](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT399E_backend.mie.ipynb)** Open In Colab + +- DTAT399F **[deeptrack.backend._config](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT399F_backend._config.ipynb)** Open In Colab + +# Developer Tutorials + +Here you find a series of notebooks tailored for DeepTrack2's developers: + +- DTDV401 **[Overview of Code Base](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/4-developers/DTDV401_overview.ipynb)** + +- DTDV411 **[Style Guide](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/4-developers/DTDV411_style.ipynb)** +# Documentation + +The detailed documentation of DeepTrack2 is available at the following link: [https://deeptrackai.github.io/DeepTrack2](https://deeptrackai.github.io/DeepTrack2) + +# Cite us! +If you use DeepTrack 2.1 in your project, please cite us: + + ``` -Benjamin Midtvedt, Saga Helgadottir, Aykut Argun, Jesús Pineda, Daniel Midtvedt, Giovanni Volpe. "Quantitative Digital Microscopy with Deep Learning." -Applied Physics Reviews 8 (2021), 011310. -https://doi.org/10.1063/5.0034891 +Benjamin Midtvedt, Saga Helgadottir, Aykut Argun, Jesús Pineda, Daniel Midtvedt & Giovanni Volpe. +Applied Physics Reviews, volume 8, article number 011310 (2021). ``` See also: -: + ``` -Midtvedt, B., Pineda, J., Skärberg, F. et al. -"Single-shot self-supervised object detection in microscopy." -Nat Commun 13, 7492 (2022). +Deep Learning Crash Course +Benjamin Midtvedt, Jesús Pineda, Henrik Klein Moberg, Harshith Bachimanchi, Joana B. Pereira, Carlo Manzo & Giovanni Volpe. +2025, No Starch Press (San Francisco, CA) +ISBN-13: 9781718503922 ``` -: + + ``` -Jesús Pineda, Benjamin Midtvedt, Harshith Bachimanchi, Sergio Noé, Daniel Midtvedt, Giovanni Volpe,1 and Carlo Manzo -"Geometric deep learning reveals the spatiotemporal fingerprint ofmicroscopic motion." -arXiv 2202.06355 (2022). +"Single-shot self-supervised object detection in microscopy." +Benjamin Midtvedt, Jesús Pineda, Fredrik Skärberg, Erik Olsén, Harshith Bachimanchi, Emelie Wesén, Elin K. Esbjörner, Erik Selander, Fredrik Höök, Daniel Midtvedt & Giovanni Volpe +Nature Communications, volume 13, article number 7492 (2022). ``` -: + ``` -Saga Helgadottir, Aykut Argun, and Giovanni Volpe. -"Digital video microscopy enhanced by deep learning." -Optica 6.4 (2019): 506-513. +"Geometric deep learning reveals the spatiotemporal fingerprint of microscopic motion." +Jesús Pineda, Benjamin Midtvedt, Harshith Bachimanchi, Sergio Noé, Daniel Midtvedt, Giovanni Volpe & Carlo Manzo +Nature Machine Intelligence volume 5, pages 71–82 (2023). ``` -: + ``` -Saga Helgadottir, Aykut Argun, and Giovanni Volpe. -"DeepTrack." (2019) +"Digital video microscopy enhanced by deep learning." +Saga Helgadottir, Aykut Argun & Giovanni Volpe. +Optica, volume 6, pages 506-513 (2019). ``` -## Funding +# Funding This work was supported by the ERC Starting Grant ComplexSwimmers (Grant No. 677511), the ERC Starting Grant MAPEI (101001267), and the Knut and Alice Wallenberg Foundation. From e73825103d3072448eb114836203802bae05a538 Mon Sep 17 00:00:00 2001 From: Alex <95913221+Pwhsky@users.noreply.github.com> Date: Mon, 10 Mar 2025 19:42:34 +0100 Subject: [PATCH 2/4] Update README.md --- README.md | 352 ++++++++++++++++-------------------------------------- 1 file changed, 106 insertions(+), 246 deletions(-) diff --git a/README.md b/README.md index 1ad6bf944..e9a471ea7 100644 --- a/README.md +++ b/README.md @@ -1,312 +1,172 @@ -

- +

- -

DeepTrack2 - A comprehensive deep learning framework for digital microscopy.

+

A comprehensive deep learning framework for digital microscopy.

- - - - - PyPI version - - - - - - Python version - - - + + PyPI version + PyPI version + Python version + +

Installation • - Getting Started • - Examples • - Advanced Tutorials • - Developer Tutorials • + Examples • + Basics • Cite us • License

-DeepTrack2 is a modular Python library for generating, manipulating, and analyzing image data pipelines for machine learning and experimental imaging. - -TensorFlow Compatibility Notice: -DeepTrack2 version 2.0 and subsequent do not support TensorFlow. If you need TensorFlow support, please install the legacy version 1.7. -The following quick start guide is intended for complete beginners to understand how to use DeepTrack2, from installation to training your first model. Let's get started! +We provide tools to create physical simulations of optical systems, to generate and train neural network models, and to analyze experimental data. # Installation -DeepTrack2 2.0 requires at least python 3.9. - -To install DeepTrack2, open a terminal or command prompt and run: -```bash -pip install deeptrack -``` -or -```bash -python -m pip install deeptrack -``` -This will automatically install the required dependencies. - -# Getting Started - -Here you find a series of notebooks that give you an overview of the core features of DeepTrack2 and how to use them: - -- DTGS101 **[Introduction to DeepTrack2](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/1-getting-started/DTGS101_intro.ipynb)** - - Overview of how to use DeepTrack 2. Creating images combining DeepTrack2 features, extracting properties, and using them to train a neural network. - -- DTGS111 **[Loading Image Files Using Sources](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/1-getting-started/DTGS111_datafiles.ipynb)** - - Using sources to load image files and to train a neural network. - -- DTGS121 **[Tracking a Point Particle with a CNN](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/1-getting-started/DTGS121_tracking_particle_cnn.ipynb)** - - Tracking a point particle with a convolutional neural network (CNN) using simulated images in the training process. - -- DTGS131 **[Tracking Multiple Particles with a U-Net](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/1-getting-started/DTGS131_tracking_multiple_particles_unet.ipynb)** - - Tracking multiple particles using a U-net trained on simulated images. - -- DTGS141 **[Distinguishing Particles with a U-Net](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/1-getting-started/DTGS141_distinguishing_particles_in_brightfield.ipynb)** - - Tracking and distinguishing particles of different sizes in brightfield microscopy using a U-net trained on simulated images. - -- DTGS151 **[Unsupervised Object Detection](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/1-getting-started/DTGS151_unsupervised_object_detection_with_lodestar.ipynb)** - - Single-shot unsupervised object detection a using LodeSTAR. - -# Examples - -These are examples of how DeepTrack2 can be used on real datasets: - -- DTEx211 **[MNIST](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/2-examples/DTEx201_MNIST.ipynb)** - - Training a fully connected neural network to identify handwritten digits using MNIST dataset. - -- DTEx212 **Single Particle Tracking** - - Tracks experimental videos of a single particle. (Requires opencv-python compiled with ffmpeg) - - -

- - -
- - - - - - -
- - - - - - -

- - -- DTEx213 **Multi-Particle tracking** -- - Detecting quantum dots in a low SNR image. - - -

- -
- - - - - - -
- - - - - - -

- - -- DTEx214 **Particle Feature Extraction** -- - Extracting the radius and refractive index of particles. - -- DTEx215 **Cell Counting** - - Counting the number of cells in fluorescence images. +DeepTrack 2.1 requires at least python 3.6. -- DTEx216 **3D Multi-Particle tracking** +To install DeepTrack 2.1, open a terminal or command prompt and run: - Tracking multiple particles in 3D for holography. + pip install deeptrack + +If you have a very recent version of python, you may need to install numpy _before_ DeepTrack. This is a known issue with scikit-image. -- DTEx217 **GAN image generation** +### Updating to 2.1 from 2.0 - Using a GAN to create cell image from masks. +If you are already using DeepTrack 2.0 (pypi version 0.x.x), updating to DeepTrack 2.1 (pypi version 1.x.x) is painless. If you have followed deprecation warnings, no change to your code is needed. There are two breaking changes: -Specific examples for label-free particle tracking using **LodeSTAR**: +- The deprecated operator `+` to chain features has been removed. It is now only possible using the `>>` operator. +- The deprecated operator `**` to duplicate a feature has been removed. It is now only possible using the `^` operator. -- DTEx231A **LodeSTAR Autotracker Template** +If you notice any other changes in behavior, please report it to us in the issues tab. -- DTEx231B **LodeSTAR Detecting Particles of Various Shapes** +# Examples of applications using DeepTrack -- DTEx231C **LodeSTAR Measuring the Mass of Particles in Holography** +DeepTrack is a general purpose deep learning framework for microscopy, meaning you can use it for any task you like. Here, we show some common applications! -- DTEx231D **LodeSTAR Detecting the Cells in the BF-C2DT-HSC Dataset** - -- DTEx231E **LodeSTAR Detecting the Cells in the Fluo-C2DT-Huh7 Dataset** - -- DTEx231F **LodeSTAR Detecting the Cells in the PhC-C2DT-PSC Dataset** - -- DTEx231G **LodeSTAR Detecting Plankton** - -- DTEx231H **LodeSTAR Detecting in 3D Holography** - -- DTEx231I **LodeSTAR Measuring the Mass of Simulated Particles** +
+

Single particle tracking

+

+ + +
+ Training a CNN-based single particle tracker using simulated data +
+ + + + Unsupervised training of a single particle tracker using LodeSTAR -- DTEx231J **LodeSTAR Measuring the Mass of Cells** - -Specific examples for graph-neural-network-based particle linking and trace characterization using **MAGIK**: - -- DTEx241A **MAGIK Tracing Migrating Cells** - - -

- -
- - - - - - -

- - -- DTEx241B **MAGIK to Track HeLa Cells** - -# Advanced Tutorials - -This section provides a list of advanced topic tutorials. The primary focus of these tutorials is to demonstrate the functionalities of individual modules and how they work in relative isolation, helping to provide a better understanding of them and their roles in DeepTrack2. - -- DTAT301 **[deeptrack.features](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT301_features.ipynb)** - -- DTAT306 **[deeptrack.properties](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT306_properties.ipynb)** - -- DTAT311 **[deeptrack.image](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT311_image.ipynb)** - -- DTAT321 **[deeptrack.scatterers](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT321_scatterers.ipynb)** - -- DTAT323 **[deeptrack.optics](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT323_optics.ipynb)** - -- DTAT324 **[deeptrack.holography](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT324_holography.ipynb)** Open In Colab - -- DTAT325 **[deeptrack.aberrations](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT325_aberrations.ipynb)** +

+
+ +

Multi-particle tracking

+ +

+ +
+ + + Training LodeSTAR to detect multiple cells from a single image +
+ Training a UNet-based multi-particle tracker using simulated data +

+
-- DTAT327 **[deeptrack.noises](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT327_noises.ipynb)** +

Particle tracing

-- DTAT329 **[deeptrack.augmentations](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT329_augmentations.ipynb)** +

+ +
+ + + Training MAGIK to trace migrating cells +

-- DTAT341 **[deeptrack.sequences](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT341_sequences.ipynb)** +# Basics to learn DeepTrack 2.1 -- DTAT381 **[deeptrack.math](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT381_math.ipynb)** +Everybody learns in different ways! Depending on your preferences, and what you want to do with DeepTrack, you may want to check out one or more of these resources. -- DTAT383 **[deeptrack.utils](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT383_utils.ipynb)** +## Getting-started guides -- DTAT385 **[deeptrack.statistics](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT385_statistics.ipynb)** Open In Colab +We have two separate series of notebooks which aims to teach you all you need to know to use DeepTrack to its fullest. The first is a set of six notebooks with a focus on the application. -- DTAT387 **[deeptrack.types](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT_387_types.ipynb)** Open In Colab -- DTAT389 **[deeptrack.elementwise](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT389_elementwise.ipynb)** Open In Colab +1. tracking_particle_cnn_tutorial demonstrates how to track a point particle with a convolutional neural network (CNN). +2. tracking_particle_unet_tutorial demonstrates how to track multiple particles using a U-net. +3. distinguishing_particles_in_brightfield_tutorial demonstrates how to use a U-net to track and distinguish particles of different sizes in brightfield microscopy. -- DTAT391A **[deeptrack.sources.base](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT391A_sources.base.ipynb)** Open In Colab +The second series focuses on individual topics, introducing them in a natural order. -- DTAT391B **[deeptrack.sources.folder](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT391B_sources.folder.ipynb)** Open In Colab +1. Introducing how to create simulation pipelines and train models. +2. Demonstrating data generators. -- DTAT391C **[deeptrack.sources.rng](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT391C_sources.rng.ipynb)** Open In Colab +## DeepTrack 2.1 in action -- DTAT393A **[deeptrack.pytorch.data](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT393A_pytorch.features.ipynb)** Open In Colab +Additionally, we have seven more case studies which are less documented, but gives additional insight in how to use DeepTrack with real datasets -- DTAT393B **[deeptrack.pytorch.features](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT393B_pytorch.data.ipynb)** Open In Colab +1. [MNIST](examples/paper-examples/1-MNIST.ipynb) classifies handwritted digits. +2. [single particle tracking](examples/paper-examples/2-single_particle_tracking.ipynb) tracks experimentally captured videos of a single particle. (Requires opencv-python compiled with ffmpeg to open and read a video.) +3. [single particle sizing](examples/paper-examples/3-particle_sizing.ipynb) extracts the radius and refractive index of particles. +4. [multi-particle tracking](examples/paper-examples/4-multi-molecule-tracking.ipynb) detects quantum dots in a low SNR image. +5. [3-dimensional tracking](examples/paper-examples/5-inline_holography_3d_tracking.ipynb) tracks particles in three dimensions. +6. [cell counting](examples/paper-examples/6-cell_counting.ipynb) counts the number of cells in fluorescence images. +7. [GAN image generation](examples/paper-examples/7-GAN_image_generation.ipynb) uses a GAN to create cell image from masks. -- DTAT395 **[deeptrack.extras.radialcenter](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT395_extras.radialcenter.ipynb)** Open In Colab +## Model-specific examples -- DTAT399A **[deeptrack.backend.core](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT399A_backend.core.ipynb)** Open In Colab - -- DTAT399B **[deeptrack.backend.pint_definition](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT399B_backend.pint_definition.ipynb)** Open In Colab - -- DTAT399C **[deeptrack.backend.units](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT399C_backend.units.ipynb)** Open In Colab - -- DTAT399D **[deeptrack.backend.polynomials](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT399D_backend.polynomials.ipynb)** Open In Colab - -- DTAT399E **[deeptrack.backend.mie](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT399E_backend.mie.ipynb)** Open In Colab - -- DTAT399F **[deeptrack.backend._config](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT399F_backend._config.ipynb)** Open In Colab +We also have examples that are specific for certain models. This includes +- [*LodeSTAR*](examples/LodeSTAR) for label-free particle tracking. +- [*MAGIK*](examples/MAGIK) for graph-based particle linking and trace characterization. -# Developer Tutorials +## Documentation +The detailed documentation of DeepTrack 2.1 is available at the following link: https://deeptrackai.github.io/DeepTrack2 -Here you find a series of notebooks tailored for DeepTrack2's developers: +## Video Tutorials -- DTDV401 **[Overview of Code Base](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/4-developers/DTDV401_overview.ipynb)** +Videos are currently being updated to match with the current version of DeepTrack. -- DTDV411 **[Style Guide](https://github.com/DeepTrackAI/DeepTrack2/blob/develop/tutorials/4-developers/DTDV411_style.ipynb)** +## Cite us! +If you use DeepTrack 2.1 in your project, please cite us here: -# Documentation - -The detailed documentation of DeepTrack2 is available at the following link: [https://deeptrackai.github.io/DeepTrack2](https://deeptrackai.github.io/DeepTrack2) - -# Cite us! -If you use DeepTrack 2.1 in your project, please cite us: - - ``` +Benjamin Midtvedt, Saga Helgadottir, Aykut Argun, Jesús Pineda, Daniel Midtvedt, Giovanni Volpe. "Quantitative Digital Microscopy with Deep Learning." -Benjamin Midtvedt, Saga Helgadottir, Aykut Argun, Jesús Pineda, Daniel Midtvedt & Giovanni Volpe. -Applied Physics Reviews, volume 8, article number 011310 (2021). +Applied Physics Reviews 8 (2021), 011310. +https://doi.org/10.1063/5.0034891 ``` See also: - +: ``` -Deep Learning Crash Course -Benjamin Midtvedt, Jesús Pineda, Henrik Klein Moberg, Harshith Bachimanchi, Joana B. Pereira, Carlo Manzo & Giovanni Volpe. -2025, No Starch Press (San Francisco, CA) -ISBN-13: 9781718503922 +Midtvedt, B., Pineda, J., Skärberg, F. et al. +"Single-shot self-supervised object detection in microscopy." +Nat Commun 13, 7492 (2022). ``` - - +: ``` -"Single-shot self-supervised object detection in microscopy." -Benjamin Midtvedt, Jesús Pineda, Fredrik Skärberg, Erik Olsén, Harshith Bachimanchi, Emelie Wesén, Elin K. Esbjörner, Erik Selander, Fredrik Höök, Daniel Midtvedt & Giovanni Volpe -Nature Communications, volume 13, article number 7492 (2022). +Jesús Pineda, Benjamin Midtvedt, Harshith Bachimanchi, Sergio Noé, Daniel Midtvedt, Giovanni Volpe,1 and Carlo Manzo +"Geometric deep learning reveals the spatiotemporal fingerprint ofmicroscopic motion." +arXiv 2202.06355 (2022). ``` - +: ``` -"Geometric deep learning reveals the spatiotemporal fingerprint of microscopic motion." -Jesús Pineda, Benjamin Midtvedt, Harshith Bachimanchi, Sergio Noé, Daniel Midtvedt, Giovanni Volpe & Carlo Manzo -Nature Machine Intelligence volume 5, pages 71–82 (2023). +Saga Helgadottir, Aykut Argun, and Giovanni Volpe. +"Digital video microscopy enhanced by deep learning." +Optica 6.4 (2019): 506-513. ``` - +: ``` -"Digital video microscopy enhanced by deep learning." -Saga Helgadottir, Aykut Argun & Giovanni Volpe. -Optica, volume 6, pages 506-513 (2019). +Saga Helgadottir, Aykut Argun, and Giovanni Volpe. +"DeepTrack." (2019) ``` -# Funding +## Funding This work was supported by the ERC Starting Grant ComplexSwimmers (Grant No. 677511), the ERC Starting Grant MAPEI (101001267), and the Knut and Alice Wallenberg Foundation. From d0282bae7a1201513008e50643524d28618025c0 Mon Sep 17 00:00:00 2001 From: Alex <95913221+Pwhsky@users.noreply.github.com> Date: Mon, 10 Mar 2025 19:44:24 +0100 Subject: [PATCH 3/4] Update README.md --- README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/README.md b/README.md index e9a471ea7..dbc5165b5 100644 --- a/README.md +++ b/README.md @@ -101,7 +101,6 @@ We have two separate series of notebooks which aims to teach you all you need to The second series focuses on individual topics, introducing them in a natural order. 1. Introducing how to create simulation pipelines and train models. -2. Demonstrating data generators. ## DeepTrack 2.1 in action From e53e68f6f049138cc2009b4e317c6b9e40364f2f Mon Sep 17 00:00:00 2001 From: Alex <95913221+Pwhsky@users.noreply.github.com> Date: Mon, 10 Mar 2025 19:46:53 +0100 Subject: [PATCH 4/4] Logo works --- README.md | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index dbc5165b5..7f6a72bf5 100644 --- a/README.md +++ b/README.md @@ -91,16 +91,12 @@ Everybody learns in different ways! Depending on your preferences, and what you ## Getting-started guides -We have two separate series of notebooks which aims to teach you all you need to know to use DeepTrack to its fullest. The first is a set of six notebooks with a focus on the application. - +We have a set of notebooks which aims to teach you all you need to know to use DeepTrack to its fullest, which consists of four notebooks with a focus on the application. 1. tracking_particle_cnn_tutorial demonstrates how to track a point particle with a convolutional neural network (CNN). 2. tracking_particle_unet_tutorial demonstrates how to track multiple particles using a U-net. 3. distinguishing_particles_in_brightfield_tutorial demonstrates how to use a U-net to track and distinguish particles of different sizes in brightfield microscopy. - -The second series focuses on individual topics, introducing them in a natural order. - -1. Introducing how to create simulation pipelines and train models. +4. Introducing how to create simulation pipelines and train models. ## DeepTrack 2.1 in action