Running the Scanner3DTop Pipeline for Generating 3D Point Clouds¶

This pipeline combines left and right 3D point clouds to create a single, merged 3D point cloud per range. Before starting, change to master branch with git checkout master.

Pipeline Overview¶

Scanner3DTop currently uses only a single distributed program:

Program	Function	Input	Output
3D MergePly	Merges PLY files into a single 3D point cloud	`left.ply`, `right.ply`	`merged.ply`

Running the Pipeline¶

Note

At this point, we assume that the interactive “foreman” and “worker” nodes have already been setup and are running, and the pipelines have been cloned from GitHub. If this is not the case, start here.

Retrieve data¶

Navigate to your directory containing Scanner3DTop, and download the data from the CyVerse DataStore with iRODS commands and untar:

cd /<personal_folder>/PhytoOracle/FlirIr
iget -rKVP /iplant/home/shared/phytooracle/season_10_lettuce_yr_2020/level_0/Scanner3DTop/<Scanner3DTop-date.tar>
tar -xvf <Scanner3DTop-date.tar>

Note

For a full list of available unprocessed data navigate to https://datacommons.cyverse.org/browse/iplant/home/shared/phytooracle/season_10_lettuce_yr_2020/level_0/Scanner3DTop/

Edit scripts¶

process_one_set.sh

Find your current working directory using the command pwd Open process_one_set.sh and copy the output from pwd into line 12. It should look something like this:
```
HPC_PATH="xdisk/group_folder/personal_folder/PhytoOracle/Scanner3DTop/"
```
Set your .simg folder path in line 13.
```
SIMG_PATH="/xdisk/group_folder/personal_folder/PhytoOracle/singularity_images/"
```

run.sh.main

Paste the output from pwd into line 7. It should look something like this:

PIPE_PATH="/xdisk/group_folder/personal_folder/PhytoOracle/Scanner3DTop/"

Set your .simg folder path in line 8.

SIMG_PATH="/xdisk/group_folder/personal_folder/PhytoOracle/singularity_images/"

In line 4, specify the <scan_date> folder you want to process. For our purposes, this will look like:
```
DATE="<scan_date>"
```

In lines 16 and 19, specify the location of CCTools:

/home/<u_num>/<username>/cctools-<version>-x86_64-centos7/bin/jx2json

and

/home/<u_num>/<username>/cctools-<version>-x86_64-centos7/bin/makeflow

Run pipeline¶

Begin processing using:

./run.sh.main

Note

This may return a notice with a “FATAL” error. This happens as the pipeline waits for a connection to DockerHub, which takes some time. Usually, the system will fail quickly if there is an issue.

If the pipeline fails, check to make sure you have a “/” concluding line 14 of process_one_set.sh. This is one of the most common errors and is necessary to connect the program scripts to the HPC.