Skip to content

Resolve "update automatic visit/subject importer"

Piotr Gawron requested to merge 334-update-automatic-visit-subject-importer into master

It's still work in progress - the changes are applied only for automatic visit importer.

This merge request creates a GUI interface for configuring and running automatic visit import. On edit study page there is section: image

Normally you would have here also import subjects, export visits, export subjects.

Currently we process only csv file. You can define column name mapping. Visit number is consistent with the whole project configuration (if visit number starts from 0 in configuration the visit numbers in csv file should also start from 0)

The file mentioned in the importer must be just a file name (not relative/absolute path) that is located in ETL_ROOT defined in local_settings.py. Files outside of the defined directory should never be processed (it's security issue - you could read OS files).

There are some difference in running importer as a cron job and from GUI:

  • when running as a cron job there is email send with the results just with the summary of the import changes, when running from GUI summary is presented as a message and all warnings are added to the message list (warnings might contain sensitive information this is why they are not send to the user)
  • cron job creates a backup of processed file and deletes it, but import from console does not touch the file (you might want to import the file to check if configuration is fine, tune the configuration and redo it)

image

The code is refactored in a way that it would be easy to extend it so many input files could be added in the future. Currently it's not possible because django_cron crates a schedule for Class type registered in setings.py not for objects created dynamically. And we cannot dynamically create schedules. This would also be a problem if we have more than one study.

Closes #334 (closed), #355 (closed)

Edited by Piotr Gawron

Merge request reports