Genie Discord forum

Author Avatarknello99
1/27/2023, 7:44:06 PM

Hi again, I am trying to design a web app that runs a psychophysics experiment and saves the experimental data. I made a previous post about using non-relational databases because I thought that would be the right database strategy, but I have since realized that using a PostgreSQL would be a lot better for my purposes, as it can store array data in columns that will be easier to access than within JSON files. I am wondering if anyone knows of/has an example of using a psql database with SearchLight specifically to save arrays of data. I think I’m missing something foundational in my approach. Basic psql documentation seems to show that array columns can be created by appending [], e.g., int[][], in the CREATE TABLE script, and data can be inserted by ‘{{1, 2},{3, 4}}’ but I was unable to find anything analogous in SearchLight. Is the only option to store everything as strings then convert to the desired datatype when processing it in Julia? If so, is there any advantage to using PostgreSQL over other SQL dbs? Below is my up() function. When running first_block = Block() and save!(first_block), throws the error “MethodError: Cannot convert an object of type Vector{SQLInput} to an object of type SQLInput”. stim_matrix and responses are AbstractArray{Real} and AbstractVector{Integer}, respectively, in the associated Blocks struct.

function up()
  create_table(:blocks) do
    [
      pk()
      columns([
        :stim_matrix => :float,
        :responses => :int,
        :stimgen => :string,
        :min_freq => :float,
        :max_freq => :float,
        :duration => :float,
        :n_trials => :float,
        :fs => :float,
        :n_bins => :int,
        :min_bins => :int,
        :max_bins => :int,
      ])
    ]
  end

  add_indices(:blocks, :stimgen, :min_freq, :max_freq, 
  :duration, :n_trials, :fs, :n_bins, :min_bins, :max_bins)
end

Thank you!

Author AvatarPere
1/30/2023, 6:33:37 PM

can you share the definition of Block as well?

Author Avatarknello99
1/30/2023, 8:04:02 PM

Yes, sorry. The default values for stim_matrix and responses are just made up for now to test. They'll initially start empty and stim_matrix will be populated on the back end while responses will be collected from the user. Here it is:

@kwdef mutable struct Block{T, W} <: AbstractModel where {T<:Real, W<:Integer}
  id::DbId = DbId()
  stim_matrix::AbstractArray{T} = [1.0 2.0 3.0; 4.0 5.0 6.0]
  responses::AbstractVector{W} = [1, 2, 3, 4]
  stimgen::String = "TestStim"
  min_freq::T = 100.0
  max_freq::T = 13e3
  duration::T = 0.5
  n_trials::W = 2000
  fs::T = 44100.0
  n_bins::W = 100
  min_bins::W = 20
  max_bins::W = 30
end
Author AvatarPere
1/30/2023, 10:00:17 PM

also, sorry forgot to ask, can you post the full error trace to see where the error is happening

Author Avatarknello99
1/31/2023, 4:05:22 AM

Here is a screenshot of the stacktrace. The db is running and connected, and first_block = Block().

Author AvatarPere
2/1/2023, 5:05:54 PM

I don't know about this one, @abhimanyuaryan can you take a look?

Author Avatarabhimanyuaryan
2/3/2023, 7:44:31 AM

@knello99 can you pass your app via zip or git link? It's getting harder to debug what's going on here. Thanks

Author Avatarknello99
2/3/2023, 7:38:10 PM

Sure, but I'm not sure it would be much help. This issue came up after doing the most basic Genie app setup and adding the code I attached here. I also don't think it's particularly surprising that trying to save an Array is throwing an error like that, since nothing is set up to store such a data type. I'm wondering if I am missing something conceptually that is pointing me down the wrong path from the start, as my programming experience is pretty much all in computational research work and this is my first foray into webdev. It must not be an uncommon need to save data in array-like structures, right? Are there other ways to go about it? I could, for example, convert the arrays to strings with specific delimiters to keep track of the shape, but I don't have a good sense of if that's a reasonable way to solve this problem (will there be any loss of precision, for example?).

Author Avatarabhimanyuaryan
2/6/2023, 5:00:16 AM

the problem is I don't know what is inside first_block so I can't debug it

Author Avatarknello99
2/7/2023, 3:36:16 PM

Here's a zip file of the app at the time I wrote the initial post. I really appreciate you taking the time to look at it! I know it's a considerable time investment.

Author Avatarknello99
2/13/2023, 4:39:20 AM

Hi, just following up on this. I tried to do more research on the topic but didn't hit anything that really addressed my issue. Maybe searchlight just doesn't have the feature yet?

Author Avatarabhimanyuaryan
2/13/2023, 2:16:17 PM

yes I also checked the api. Looks like SL doesn't support this