Extract attributes from classes to build statistics files

[ns-rse]

We are almost at the point of switching to the new [Data]Class based objects which negates the need to pass around
dataframes of statistics between different steps of the analysis as these are attributes of the dataclass objects (or
can be obtained from the attributes).

This simplifies the process of analysing objects going forward as we don't need to worry about passing these around with
the underlying images to be processed.

However we still want to have dataframes that we can write to .csv at the end.

By using dataclasses we get __dict__() of all attributes for classes for free.

To simplify making the desired dataframes we can leverage this and compile a nested dictionary from the TopoStats
class down to the Molecule class of just the attributes we want.

Perhaps abstracting it out to a recursive function that traverses through an object/dataclass until it gets to something with data to extract at which point it calls __dict__() and takes a subset of the attributes (aka statistics) to return as part of the dictionary.

At the end we can leverage pd.DataFrame.from_dict() to build the resulting statistics dataframe for each image and
molecules within it and then pd.concat() these after all images have been processed to give the desired data to write
to CSV.

[ns-rse]

From @tobyallwood

Could you clarify what you mean in when talking about only getting attributes we want? I understand the concept of the function looping through nested classes until actual data is found (converting it all into a DataFrame) but how would a user/ outside function define what data it specifically wants? For now I will build the function to return all data (inc. nested class's data as a dict)

Sorry for the lack of clarity here @tobyallwood, many of the classes have Numpy arrays as attributes and these need to be accessible as they are what the processing is performed on but they are also included when we call __dict__().

However, for producing summary statistics we don't need these Numpy arrays, we need the values that are to go into the various CSV files (i.e. grain_statistics.csv / image_statistics.csv / molecule_statistics.csv).

As such when building the nested dictionary I'm suggesting here we should only select the statistics we want from the dictionary returned by __dict__() and disregard those items we don't want/need such as Numpy arrays.

I hope that clarifies my thinking but don't hesitate to ask if its still not clear.

[tobyallwood]

Aah ok yeah that makes sense thanks @ns-rse. Would it be as simple as filtering by data type or would there be edge cases (e.g. floats that aren't needed in output CSVs) that would require more specific code? Right now my thinking is I'll go through the CSVs to confirm what would need exporting but let me know if you know already.

[ns-rse]

Yes, I'd be guided by the CSVs 👍