Custom analysis of hdf5 files

In this notebook we add features to the h5 files we converted the previous notebook 1.

ndf_folderpath = '/media/jonathan/DATA/seizure_data/test_ndf'

ndf_filepaths = [os.path.join(ndf_folderpath,f) for f in os.listdir(ndf_folderpath)]

test_ndf = ndf_filepaths[0]
test_ndf

ndf_file = pg.NdfFile(test_ndf, fs = 'auto', verbose=True)
ndf_file.get_valid_tids_and_fs()
ndf_file.tid_set

ndf_file.tid_to_fs_dict

ndf_file.load(read_ids=[88])

ndf_file[88]

### Optional: Loading the h5 files:

At this point you can load the h5 files in the gui and manually inspect them for seizures or activity of interest.

You can also load them in python notebooks for further analysis. Here you have two main options, either use the H5File class from pyecog, or work directly with the h5 file with the h5py module.

h5_filepaths = [os.path.join(h5_folderpath,f) for f in os.listdir(h5_folderpath)]
h5_filepaths[0]

Loading the pyecog module

The easiest place to place and run this notebook is from the pyecog directory downloaded from github, e.g. “pyecog-Development” as the pyecog module will be found in this folder. However, if you want to run the notebook from else where on your computer you first need to make sure that python can find the pyecog module using sys.path.append(). To do this modify and copy the following code into a cell and run it (shift+enter).

import sys
pyecog_path = 'home/jonathan/git_repos/pyecog' # replace this with the pyecog location
sys.path.append(pyecog_path)

If you are on windows you have to deal with the problem that backslashes in your paths are treated escape characters by python. Prefixing the string with ‘r’ prevents this.

pyecog_path_windows = r'home\jonathan\git_repos\pyecog' # replace this with the pyecog location

import sys
import os

# if you are in the directory downloaded from github you do not have to run this cell

pyecog_path = '/home/jonathan/git_repos/pyecog' # replace this with the Pyecog-Master location
sys.path.append(pyecog_path)

import pyecog as pg

pg # check the module is imported from where you expect

<module 'pyecog' from '/home/jonathan/git_repos/pyecog/pyecog/__init__.py'>

The h5 folder is where we stored the h5 files that we converted in notebook 1.

h5_folderpath  = '/media/jonathan/DATA/seizure_data/test_h5'

Optional: Loading the h5 file features:

At this point you can load the h5 files in the gui and manually inspect them for seizures or activity of interest.

You can also load them in python notebooks for further analysis. Here you have two main options, either use the H5File class from pyecog, or work directly with the h5 file with the h5py module.

h5_filepaths = [os.path.join(h5_folderpath,f) for f in os.listdir(h5_folderpath)]
h5_filepaths[0]

'/media/jonathan/DATA/seizure_data/test_convert_h5/M1518706020_2018-02-15-14-47-00_tids_[88, 89, 90, 91, 92, 94, 104].h5'

h5_file = pg.H5File(h5_filepaths[0])

h5_file.attributes

{'Mcode': 'M1518706020',
 'fs_dict': '{104: 256, 88: 256, 89: 256, 90: 256, 91: 256, 92: 256, 94: 256}',
 'num_channels': 7,
 't_ids': array([ 88,  89,  90,  91,  92,  94, 104])}

h5 file objects can be indexed by their transmitter id number, this will return a dictionary for that transmitter.

h5_file[88]['features_df']

	min	max	mean	std-dev	kurtosis	skew	sum(abs(d	bl_len	1-4 Hz	4-8 Hz	8-12 Hz	12-30 Hz	30-50 Hz	50-70 Hz	70-120 Hz
0	-2.745696	3.921481	0.001907	0.951816	0.562453	0.154258	507.375421	1280.0	0.276522	0.282955	0.058782	0.137594	0.073938	0.037437	0.023082
1	-2.538420	2.875458	-0.011958	0.918251	-0.055107	0.012742	484.469184	1280.0	0.297290	0.160897	0.037914	0.177929	0.056228	0.041208	0.021928
2	-3.369785	4.830944	-0.005697	1.057422	1.490759	0.513808	497.692428	1127.0	0.326884	0.306726	0.120561	0.212007	0.057005	0.036853	0.020290
3	-4.484467	3.005027	0.018585	1.049311	0.416406	-0.244072	518.875038	1265.0	0.441402	0.255402	0.077621	0.211842	0.080629	0.035725	0.020244
4	-9.822171	3.620097	0.002888	1.051635	5.856197	-0.600460	560.345812	1256.0	0.258222	0.304614	0.085988	0.262057	0.107011	0.050286	0.075028
5	-5.016475	3.796506	-0.010853	1.094132	1.413410	0.009865	528.680387	1195.0	0.466390	0.289356	0.083845	0.208914	0.092203	0.040994	0.028764
6	-2.360754	3.465912	-0.014044	0.972277	0.316050	0.405794	493.823822	1211.0	0.470302	0.165675	0.060854	0.185965	0.055256	0.038203	0.021816
7	-3.784373	3.380679	0.018908	1.188957	-0.070337	-0.157187	510.652488	1224.0	0.527945	0.429023	0.124392	0.208431	0.072418	0.040214	0.019761
8	-7.965405	5.122307	-0.006857	1.294961	4.149093	-0.262928	544.083040	1047.0	0.711421	0.348831	0.158709	0.280185	0.097763	0.040309	0.020178
9	-5.229404	6.754734	-0.009964	1.275355	1.278787	0.221219	513.431716	1111.0	0.711461	0.334720	0.143933	0.369128	0.101415	0.038544	0.021516
10	-3.854198	5.586577	-0.005269	1.284305	0.921093	0.305607	482.475379	1104.0	0.800779	0.268881	0.173424	0.293042	0.060466	0.029142	0.014935
11	-3.525886	3.773122	0.017021	1.297012	-0.131703	0.134103	472.318159	1121.0	0.802087	0.324443	0.114645	0.210437	0.074116	0.033903	0.016987
12	-5.649103	15.035132	-0.019014	1.549661	11.755122	1.285502	530.833004	978.0	0.531510	0.745433	0.400011	0.441124	0.093635	0.056163	0.018926
13	-4.956014	3.414950	0.044344	1.316174	0.474763	-0.407176	481.310986	1114.0	0.864683	0.297260	0.150904	0.275476	0.065509	0.032104	0.013759
14	-3.954851	4.646735	-0.008864	1.398841	0.419287	0.133754	515.690621	1073.0	0.830538	0.298522	0.187436	0.304622	0.082487	0.029620	0.016897
15	-3.486569	4.004167	-0.002922	1.193170	0.449053	0.017427	450.833731	1128.0	0.533526	0.447185	0.164795	0.230173	0.066359	0.027804	0.012576
16	-9.185851	4.745833	-0.020753	1.404140	1.645688	-0.104998	501.452628	833.0	0.650680	0.489238	0.168410	0.285406	0.082329	0.038631	0.023186
17	-5.260774	6.324856	0.022187	1.306987	1.604539	-0.008105	451.570608	1140.0	0.816545	0.372041	0.151846	0.312183	0.060156	0.023657	0.009984
18	-2.457274	3.264696	-0.014664	0.901966	0.426506	0.300696	421.600059	1280.0	0.224136	0.201921	0.153110	0.201433	0.064692	0.026570	0.009348
19	-3.295339	5.413536	-0.007183	1.243465	0.742899	0.733430	422.116302	1052.0	0.850298	0.197062	0.089558	0.323254	0.052151	0.020226	0.007090
20	-2.664857	3.321672	-0.004571	0.840505	0.425044	0.112048	416.961986	1280.0	0.246920	0.103103	0.058986	0.151078	0.063307	0.026897	0.012600
21	-2.877755	3.820785	0.014660	0.908444	0.463671	0.087917	388.812314	1280.0	0.309512	0.146225	0.063018	0.154775	0.051886	0.023127	0.006976
22	-2.306637	4.432575	-0.004770	1.087817	1.173868	0.669565	429.693485	1199.0	0.517649	0.232056	0.115126	0.190860	0.070079	0.020856	0.008823
23	-4.008120	3.701702	0.002494	1.112408	0.482239	0.230278	450.061029	1189.0	0.493985	0.333859	0.048737	0.215048	0.076206	0.028147	0.010693
24	-3.377793	4.059548	0.044629	1.265754	0.072259	0.265524	444.619697	1067.0	0.612792	0.497030	0.201418	0.242031	0.080960	0.027148	0.005646
25	-4.503251	5.351383	-0.045337	1.375659	0.533906	0.262147	426.677626	902.0	0.785479	0.487603	0.226601	0.264577	0.049743	0.021211	0.006866
26	-3.568531	4.336241	-0.015334	1.379952	-0.251292	0.161377	434.372402	922.0	0.928004	0.380891	0.146578	0.222205	0.058511	0.025594	0.007110
27	-4.892261	4.199362	0.025549	1.379633	0.635587	-0.130585	427.890061	857.0	0.779472	0.608448	0.116563	0.246202	0.062507	0.026885	0.006059
28	-5.911869	4.847576	-0.014171	1.496278	0.525630	0.137111	452.850210	883.0	0.910086	0.609288	0.282872	0.325669	0.064047	0.030313	0.007209
29	-4.214667	3.577526	-0.026826	1.170126	0.026517	0.048512	439.250432	1154.0	0.470861	0.355157	0.296465	0.245023	0.065580	0.020623	0.007572
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
690	-5.757212	14.496787	-0.012677	1.471006	23.335535	2.749804	440.605153	1052.0	0.779282	0.525149	0.322763	0.601284	0.085317	0.030744	0.010619
691	-4.702798	4.127438	0.015693	1.214199	0.946584	-0.211986	463.535408	1050.0	0.732721	0.360134	0.142084	0.235886	0.064723	0.030105	0.012696
692	-4.276248	6.411844	-0.014816	1.307440	1.869722	0.725547	470.026684	1084.0	0.668813	0.477376	0.184708	0.334354	0.061518	0.031714	0.010888
693	-4.342320	3.785950	-0.001922	1.335172	0.075070	0.099501	501.047427	970.0	0.747869	0.246368	0.199893	0.304325	0.069427	0.034202	0.016603
694	-3.999692	4.905349	-0.002278	1.279467	0.573413	0.380584	454.667219	1095.0	0.627531	0.425904	0.164660	0.304105	0.065938	0.025006	0.012333
695	-4.370572	5.235374	0.019616	1.646325	0.183652	0.335276	511.442734	704.0	1.302879	0.506967	0.239064	0.319867	0.067544	0.034969	0.015418
696	-4.143962	3.496515	-0.019012	1.310633	-0.307614	0.002483	478.246595	996.0	0.947350	0.505394	0.119981	0.212160	0.078751	0.026191	0.014225
697	-2.939468	3.645246	0.002312	0.989951	0.199146	0.149408	486.805228	1280.0	0.377750	0.322209	0.061835	0.138720	0.067689	0.037093	0.021757
698	-3.330699	4.829200	-0.004158	1.013271	1.775634	0.333971	469.286519	1138.0	0.357334	0.206490	0.085539	0.186886	0.059271	0.031302	0.017955
699	-4.279370	5.720537	0.001136	1.226381	1.751687	0.412864	499.944392	1049.0	0.637580	0.388617	0.133865	0.240075	0.061437	0.037101	0.019387
700	-4.073154	4.808202	0.016762	1.225125	0.120327	0.112915	517.026428	1163.0	0.701984	0.322741	0.121541	0.165357	0.078646	0.041100	0.026292
701	-4.593632	4.674571	-0.012404	1.327265	0.105494	0.184291	504.381454	1024.0	0.795026	0.384167	0.132930	0.220840	0.069287	0.038949	0.022405
702	-4.167829	4.319712	0.005672	1.308253	0.299642	0.194562	535.966277	1009.0	0.773908	0.393674	0.104382	0.237424	0.068482	0.046995	0.026336
703	-4.196323	6.620794	0.010787	1.504828	0.727306	0.311355	554.941830	871.0	1.261631	0.548588	0.151882	0.256520	0.075753	0.051780	0.023556
704	-3.955023	4.914947	-0.008554	1.320213	0.371810	0.390918	495.560486	981.0	0.857747	0.376863	0.119071	0.182949	0.063222	0.033984	0.018896
705	-3.608728	3.661762	-0.024912	1.268253	-0.194817	-0.013589	528.876712	1201.0	0.671145	0.378833	0.146457	0.234801	0.073901	0.043383	0.020795
706	-4.629775	5.489401	0.016969	1.392876	0.406126	0.138684	521.157776	962.0	0.863635	0.447052	0.223284	0.265719	0.084475	0.036293	0.020991
707	-3.569970	9.013496	0.000711	1.289536	1.906456	0.471651	523.800450	991.0	0.543882	0.469144	0.240843	0.286008	0.086718	0.045509	0.050148
708	-3.707065	4.581943	-0.002083	1.357154	-0.033649	0.194637	522.551612	1044.0	0.672098	0.350869	0.211379	0.355743	0.078794	0.033485	0.018762
709	-4.190712	5.519274	0.001844	1.299737	0.757940	0.498762	510.930416	1089.0	0.595124	0.366942	0.157764	0.326363	0.081769	0.033800	0.016922
710	-3.386481	5.271014	0.030382	1.547091	0.022169	0.513613	497.314751	735.0	1.187509	0.477200	0.255087	0.325033	0.076742	0.027689	0.013524
711	-3.699453	4.898075	-0.027988	1.364191	0.206366	0.477663	459.457768	952.0	0.926517	0.437916	0.117424	0.274607	0.059559	0.027609	0.013117
712	-4.624734	9.595704	0.001354	1.504431	2.872975	0.436785	434.575862	897.0	1.384712	0.485641	0.141521	0.240223	0.101878	0.027615	0.009540
713	-4.335477	5.717319	0.002326	1.413635	0.537906	0.344380	472.618872	905.0	1.017626	0.456291	0.135159	0.235540	0.077904	0.026629	0.012351
714	-4.467684	5.695909	-0.005388	1.511456	0.339366	0.136025	467.201641	810.0	1.299472	0.464950	0.220517	0.207499	0.065736	0.029378	0.012627
715	-3.093008	4.777498	0.024668	1.339723	0.055912	0.373936	452.100753	1035.0	0.887479	0.372953	0.147838	0.251702	0.071332	0.024734	0.008532
716	-4.280185	3.764713	-0.027456	1.212568	0.178199	-0.156122	481.772357	1060.0	0.482698	0.413855	0.255187	0.313109	0.065660	0.029568	0.009617
717	-5.789287	4.444491	-0.014628	1.302015	1.276640	-0.076348	409.601379	951.0	0.705501	0.542924	0.147105	0.234677	0.053697	0.019287	0.007973
718	-3.463968	4.674178	0.017524	1.240515	0.485109	0.291203	388.131007	1114.0	0.711683	0.480209	0.132861	0.134867	0.043589	0.021719	0.007139
719	-3.712615	6.296349	0.005182	1.360754	0.935015	0.647082	421.855224	1016.0	1.024474	0.350026	0.128241	0.222323	0.053141	0.025245	0.007555

720 rows × 15 columns