| Info | ||
|---|---|---|
| ||
This page is under construction. |
GitLab parsers are used to generate XML which allows us to import the questionnaire straight questionnaires directly into Archivist, rather than entering it them from scratch. See below for instructions on creating a GitLab account.
Note: It is important to make sure that you have the latest version of the parser every time you run it. If updates have been made, you should re-fork or pull the latest version of the parser to ensure you have the latest version. See Updating your GitLab pipeline for how to do this.
...
Creating a GitLab Account
a. If you don’t have a GitLab account, please register at https://gitlab.com/.
b. Once you sign in on GitLab go to https://gitlab.com/jli755/archivist_insert_workaround.
c. Fork the repo on Gitlab.com - click “Fork” in the right-hand corner of the window (see screenshot below).
d. You now need credentials for Heroku and Archivist login. Please contact Hayley Mills to obtain the variables.
i. First, make sure you are in the right part of the GitLab (i.e., in your account). The URL should look like: https://gitlab.com/(your account name)/archivist_insert.
ii. To add the variables, go to Settings → CI/CD → Variables → Expand → Add Variable.
Make sure both boxes (Protect variable and Mask variable) are ticked.
...
Upload tables to database via pipeline
a. Tables - You can use "csv" or "tsv" table formats, but not a mixture of them. (If you are using the "tsv" tables please see step 2 below)
i. Copy your tables into "archivist_table"
1. Open the “archivist_tables” folder in Gitlab
2. Click “upload file” to copy the files to the folder (you have to upload individual files, therefore repeat the process for to upload all of your files.
3. To stop running the pipeline automatically, you need to add info about when you add or update on of these files. Otherwise, it will automatically run the pipeline. You should only run the pipeline when all files have been added; so in order to stop it running when you are not ready add [skip ci] to the comment.
b. tsv tables instead of csv tables
i. Need to change the delimiter in the db_temp.sql file
ii. May need to specify the encoding of the file (Please see below)
| Code Block | ||
|---|---|---|
| ||
\COPY temp_sequence FROM 'archivist_tables/sequence.tsv' DELIMITER E'\t' CSV HEADER encoding 'windows-1251'; |
iii. If it passes, great. If not, look at the output cross mark to see what went wrong. Click on cross marks (1&2) on stages column (Please see the screen shot below)
...
Correct the errors on csv files
a. If the csv files have formatting problem, it will not pass stage 1 (run_tests). Make sure all the table formats are compliance with the correct format. See Tables structure
b. Having extra spaces on the uploaded table create issues. Just delete the extra spaces and run the pipeline again to fix this error.
...
Download the xml file and export it to archivist
a. If it passes, the XML file is available as an "artifact" (a zip files containing the generated XML) and this is available for 10 days. The xml can be viewed temporarily from temp archivist and will need to be loaded into archivist via import, if you have permissions to do so then add to archivist, if not, please ask Hayley Mills to import.
Updating your gitlab pipeline
Once setup you may need to re-fork or pull the latest version of the Gitlab pipeline to ensure you have the latest version.
Before you start, you will need to ensure that you have git installed.
To pull the latest version you will need to do this on the command line.
1) Search/open cmd
2) Clone your fork. You can find the address on your Gitlab fork. Copy the Clone with HTTPS link.
| Code Block |
|---|
git clone https://gitlab.com/HayleyMills/archivist_react_export.git |
This will create a folder on your current directory.
2) Go to that new folder
| Code Block |
|---|
cd archivist_react_export |
3) Connect to the main/master version. This is the project you have created your fork from. As above find the https address under clone. Follow the example below to pull this and then push this back to your version. You may need to enter your Gitlab email and password. Note the example below uses main for this, but if this does not work try master instead.
| Code Block |
|---|
git remote add upstream https://gitlab.com/jli755/archivist_react_export.git
git checkout main
git fetch upstream
git pull upstream main
git push origin main |
4) Your Gitlab fork will now have been updated to the latest version. Check that it is merging the main branch and that the history matches the main version.
Trouble shooting:
...
Underneath this page, you can find information on creating a GitLab account and running the archivist_insert parser.