Especially while migrating large databases, the scripts have no user feedback implemented. If the data warehouse host interrupts there will be no feedback. Additionally it is helpful to have a approximation of calculation time.
Task: Add a loading feature
Display loading progresses for each individual step and
display the overall progress (search for a progress marker that works for postgres as well)
There are apt libraries that can be used, like pv (pipe viewer), to show progress by file size:
# install pipe viewer
sudo apt-get install pv
# example usage
sudo docker exec -i "$docker_db_container" \
bash -c "pv /backup/backup_i2b2.sql | psql -U postgres -d i2b2 -q"
Especially while migrating large databases, the scripts have no user feedback implemented. If the data warehouse host interrupts there will be no feedback. Additionally it is helpful to have a approximation of calculation time.
Task: Add a loading feature
Display loading progresses for each individual step and
display the overall progress (search for a progress marker that works for postgres as well)
There are apt libraries that can be used, like pv (pipe viewer), to show progress by file size: