Logs

Gravity generates logs for each execution of a job. These logs can be helpful for:
investigating failed executions
viewing the statuses of past executions
viewing the status and current stage of a recently executed job
validating execution run times
There are two ways to view execution logs:
The logs page - View the history of all jobs that have been executed. You can apply filters and grouping when navigating the logs. You can also dive into the execution trace of a log by clicking on the
button.
The jobs page - View the details about the last execution of each job. You will see the status of the execution and, in the case of running executions, you can see the current stage. If the last job execution finished with an error, you can investigate by clicking the
button to view the execution trace. You can also view all historical executions for a given job by clicking on the shortcut
to the logs page where the relevant filters for the job are pre-applied.
click the shortcut to view all logs for the job
hover over the information icon to see the real-time stage of a running job
Job Execution Status
A job execution can have one of the following statuses:
RUNNING - Job execution is currently running
FINISHED - Job execution has finished successfully
ERROR** ****** - Job execution has finished with errors
These statuses apply to each job execution as well as each child stage of the job execution. The statuses of each child stage are available in the execution trace.
Execution Trace
The execution trace can be accessed by clicking on the 
 button. This is available on both the Jobs and on the Logs page. The execution trace will provide the status of the parent and child stages. For running executions, the current stage will be displayed and updated in real time as the execution progresses.
Execution Trace - Tree View
Stage of a running execution
The following options are available to help navigate the execution trace:
​
 expand the JSON tree to see all of the child stages of the job
​
 collapse the JSON tree to hide all child stages of the job
​
 navigate directly to the error when the JSON tree is expanded
​
 -refresh the execution trace, if you are investigating a job that is running. The trace is not updated automatically as it is difficult to read changing logs.
​
 - rerun the job
Key Execution Stages
The execution stages for a job will vary depending on the source, target and job configuration. The table below highlights some key stages of a job that is configured to load data into a database target
Stage
Description
fetch job configuration
Get the configuration for the job that is being run
prepare target DB
Check that the target credentials are still valid. If the target table already exists, check that the schema is correct
execute source preparation
Prepare to extract data from the source. This can include compiling credentials, extracting and reading metadata, determining increments etc.
execute source processing
Extract data from the source
create target table
Create the the target table and staging area if they do not already exist.
load data to target
Load data to a staging area in the database
merge data to target table
Merge, append or replace the data in the target table
​​​​

Stage	Description
fetch job configuration	Get the configuration for the job that is being run
prepare target DB	Check that the target credentials are still valid. If the target table already exists, check that the schema is correct
execute source preparation	Prepare to extract data from the source. This can include compiling credentials, extracting and reading metadata, determining increments etc.
execute source processing	Extract data from the source
create target table	Create the the target table and staging area if they do not already exist.
load data to target	Load data to a staging area in the database
merge data to target table	Merge, append or replace the data in the target table

Last modified 9mo ago