Skip to content

A collection of bash (and some matlab) scripts to automatically carry out much of the data processing pipeline on nested folders containing Trodes data files. It basically crawls recursively through a folder structure and carries out Export functions (on multiple cores as background shell processes) and TrodesToMatlab on each processable datum.

License

Notifications You must be signed in to change notification settings

JadhavLab/TrodesDataProcess

Repository files navigation

INSTALLATION INSTRUCTIONS


Note: Linux only at the moment -- Mac OS fixes coming soon.

  1. type "which realpath" ... if nothing comes up, its a quick download. As long as you have unix, do the following

    a) Debian/Ubuntu: sudo apt-get install realpath b) Mac OS: (SCRIPTS NOT WORKING FOR MAC AT THE MOMENT) If you do not have macports, install it; then run sudo port install realpath

  2. Enter the path to the folder containing your export functions (e.g. exportspikes, exportdio) in TrodesPath.txt e.g. "~/Code/Trodes/Export/"

  3. For automated matclust and NeuroQuery file creation, have TrodesToMatlab and BashTrodes folders/subfolders in path

  4. DONE!

PURPOSE


This script suite enables one to process an entire server or folder structure containing one or multiple experiments, each having their own session folders of rec, trodeComment, stateScriptLog, and videoPositionTracking files, in one or two commands.

  1. SetConfiguration.sh

    Usage: ./SetConfiguration.sh X Y , where X = recursive folder path Y = configuration file

SetConfiguration permits you to take a configuration file (.xml or .trodesconf) and apply it to every rec file in a directory structure. The script dives into each folder and for every X.rec it finds, it creates a X.trodesConf with the Y configuration file provided.

  1. SingleProcess.sh

    Usage: ./SingleProcess.sh X Y Z , where X = recursive folder path Y = exportdio or exporttime or exportspikes or exportLFP Z = filter string, e.g. SJ6

SingleProcess can apply any Trodes export function Y to every .rec it finds in the entire nested folder structure of X. Z is an optional filter argument, and the script will only process files partially matching filter string Z. E.g. if it's "DM4", only rec files beginning with DM4 will be processed. When the program finds a folder of rec files, it stitches them together into a single resultant output, just like the matlab export functions. Further, it titles the output with the largest string common in all matching rec file names. E.g. DM4_Wmaze1.rec DM4_WMaze2.rec would produce a folder for the the output as DM4_WMaze.*

  1. AllProcess.sh

    Usage: ./AllProcess.sh X Y , where X = recursive folder path Y = filter string, e.g. SJ6

Applies all export programs to all recfiles in the nested folder structure of X. Knocks out all of the extraction process once and for all, generating all the .dat files for all .rec files. This opens 4 processes that run in the background and handle each of the exports respectively. Once this process has begun, to terminate, type "q" and hit enter.

The program proceeds to call MatclustAndFilter.m on all files that it finished processing, and places your filter files into folders in accordance with your animaldef.m

  1. MatclustAndFilter.m

    Usage: (cd to the folder structure of interest) ... then type into Matlab "MatclustAndFilter"

This matlab function finds every .LFP, .spikes, .time, .DIO folder nested subfolders and process them to genereate matclust and NeuroQuery (Filter Framework) files. In order to assign a session number automatically, it only has expectation. Every time it finds a .DATA folder, it looks at the parent folder to infer its session ID number. So, suppose you have X.LFP, it then looks at the folder structure above it, and per folder containing one single session, it expects a date or session number and date in the folder name. For example, X.LFP may exist in 20160204, and the folder that contains that folder has sessions 20160203 20160204 20160205. It then infers that the session ID of X.LFP is 2. It does this by ranking dates it pulls out and computing its location in that list. ALternatively, one can structure folders containing each session as 01_date1 02_date2 03_date2 04_date3 and so on, and it will ignore the date, and interpet the number leading the underscore as session number. All other folders in the hierarchy can be titled however.

About

A collection of bash (and some matlab) scripts to automatically carry out much of the data processing pipeline on nested folders containing Trodes data files. It basically crawls recursively through a folder structure and carries out Export functions (on multiple cores as background shell processes) and TrodesToMatlab on each processable datum.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 3

  •  
  •  
  •