Genie Discord forum

Hi all,

I am at the very beginning of using Genie to set up a psychophysics experiment that my lab has been developing. We are planning to host it on AWS. I am looking for some database setup advice. Upon completion of the experiment, every user will have an approximately 32x800 (exact size unknown until they begin the experiment, as it is based on selected settings) matrix, a vector whose length is the second dimension of the matrix, and some configuration settings that need to be saved. From what I’ve learned so far, I do not think a relational database such as those that SearchLight supports is the correct choice for this data. I am very new to this type of work though, so I would appreciate any thoughts on good storage methods for this data and any Julia packages that can be integrated with Genie to support it.

Thank you!

I'm not an expert, but I'd say it depends on your code. If you're moving around a struct that has the matrix, vector and configuration as fields then it might make sense to use an ORM. Or even if they're separate structs. It's just that it could be easier to use JLD2 and JSON files if you don't need to do searches, filtering or other DB stuff

I'm working on a database example where I'm predicting the value of houses stored in a database. I'm constantly loading House objects from the DB to update their value predictions and save them back to the DB, and the ORM comes in very handy since i don't have to write any SQL. You can take a look at the code, especially the app.jl where the load and save operations are performed. Sorry that it's not documented, I'll do it soon 🙂

https://github.com/GenieFramework/GenieFrameworkDemos_NewAPI/blob/main/Database/

Thank you for the suggestions! I think a JSON storage method might be just the thing, since the size of the matrices and vectors (really, it could be stored as vectors and integers which are then concatenated) can be such a wide range, and they only ever go together as pairs. I also won’t need to access individual indices. The most preprocessing before analysis I think I will need to do is to combine multiple groups of vectors and matrices by matching user_ID and experiment_ID fields.

Do you know of any examples of storing JSON files with SearchLight? Is it as simple as creating the file and storing it in a column?