Tsne file

Data_wide_to_long.py

@@ -0,0 +1,53 @@
+import pandas as pd
+import numpy as np
+import sys
+
+
+path = r'C:\Users\andre\Desktop\FFR_data\Pressure_tracing-deID.csv'
+df = pd.read_csv(path)
+
+
+# create a unique ID
+df['id'] = df.index
+
+# rename columns
+col = ['ID', 'Pos_Neg', 'Def']
+for i in range(0, len(df.iloc[0, 3:-1])):
+    col.append('index{var}'.format(var=i))
+col.append('id')
+df.columns = col
+
+# convert data into long format
+df = pd.wide_to_long(df, stubnames='index', i='id', j='col')
+
+# create separate col for each observation
+df_final = pd.DataFrame()
+df_id = df.groupby(['ID'])
+for i, row in enumerate(df_id):
+    df_def = row[1].groupby(['Def'])
+    if i == 0:
+        for j, col in enumerate(df_def):
+            if j == 0:
+                cols_new = pd.DataFrame({'ID': list(col[1].loc[:, 'ID']), 'Pos_Neg': list(col[1].loc[:, 'Pos_Neg'])}, columns=['ID', 'Pos_Neg'])
+                df_final = pd.concat([df_final, cols_new], axis=1)
+                HR = pd.DataFrame(list(col[1].iloc[:, 3]), columns=[str(col[0])])
+                df_final = pd.concat([df_final, HR], axis=1)
+            else:
+                new_obs = (pd.DataFrame(list(col[1].iloc[:, 3]), columns=[str(col[0])]))
+                df_final = pd.concat([df_final, new_obs], axis=1)
+    else:
+        df_hold = pd.DataFrame()
+        for j, col in enumerate(df_def):
+            if j == 0:
+                cols_new = pd.DataFrame({'ID': list(col[1].loc[:, 'ID']), 'Pos_Neg': list(col[1].loc[:, 'Pos_Neg'])}, columns=['ID', 'Pos_Neg'])
+                df_hold = pd.concat([df_hold, cols_new], axis=1)
+                HR = pd.DataFrame(list(col[1].iloc[:, 3]), columns=[str(col[0])])
+                df_hold = pd.concat([df_hold, HR], axis=1)
+            else:
+                new_obs = (pd.DataFrame(list(col[1].iloc[:, 3]), columns=[str(col[0])]))
+                df_hold = pd.concat([df_hold, new_obs], axis=1)
+        df_final = pd.concat([df_final, df_hold], axis=0)
+df_final = df_final.dropna()
+df_final.to_csv(r'C:\Users\andre\Desktop\FFR_data\Tracing_long_AC.csv')
+
+

README.md

@@ -1,2 +1,25 @@
 # python_journey
 learning python
+These codes are part of my journey learning python. 
+
+The first part focusses on geospatial analysis and use the osmnx library. The tutorial paper is available [here](https://www.frontiersin.org/articles/10.3389/fneur.2019.00743/full). The [Brisbane.ipynb](./Brisbane.ipynb) python notebook deals with journey from Westin Hotel to surrounding building within 500 metres radius. A second example uses codes provided from osmnx for examining street network orientation in Australian Capital Cities. The AustCities-Copy1.ipynb note book is available in gh-pages. [![here](./AustCities.png)](./AustCities-Copy1.ipynb) 
+
+The second part focusses on machine learning and use sklearn. Random forest is , a supervised machine learning method related to decision tree analysis, which employed random selection of covariates and patients from the dataset to create multiple trees. This form of ensemble learning utilises ‘wisdom of the crowd’ to create the model. This example illustrates the use of regression with random forest and a plot of observed vs predicted is provided in RFstandfirm.ipynb notebook. An example with random forest classification is provided in the DrivingReg.ipynb notebook. This example comes with demo of creation of GUI for testing new data using the model created by random forest. 
+
+[![here](./RFstandfirm_regression.png)](./RFstandfirm.ipynb)
+
+The third part focusses on manifold learning and uses sklearn manifold tsne. The T-distributed Stochastic Neighbor Embedding example is provided below. This is a low dimensional representation of the thrombectomy trial data. The file also contains illustration about merging pandas data frame and provides 3 types of plot: seaborn, matplotlib and plotnine-ggplot style.
+
+The seaborn plot is shown. 
+
+[![here](./TSNEecr_sns.png)](./TSNEecr.ipynb)
+
+The matplotlib plot is shown.
+
+![here](./TSNEecr_matplotlib.png)
+
+The plotnine plot is shown. 
+
+![here](./TSNEecr_plotnine.png)
+
+