Logs
Gravity generates logs for each execution of a job. These logs can be helpful for:
investigating failed executions
viewing the statuses of past executions
viewing the status and current stage of a recently executed job
validating execution run times
There are two ways to view execution logs:
Job Execution Status
A job execution can have one of the following statuses:
RUNNING - Job execution is currently running
FINISHED - Job execution has finished successfully
ERROR** ****** - Job execution has finished with errors
These statuses apply to each job execution as well as each child stage of the job execution. The statuses of each child stage are available in the execution trace.
Execution Trace
The following options are available to help navigate the execution trace:
Key Execution Stages
The execution stages for a job will vary depending on the source, target and job configuration. The table below highlights some key stages of a job that is configured to load data into a database target
Stage | Description |
---|---|
fetch job configuration | Get the configuration for the job that is being run |
prepare target DB | Check that the target credentials are still valid. If the target table already exists, check that the schema is correct |
execute source preparation | Prepare to extract data from the source. This can include compiling credentials, extracting and reading metadata, determining increments etc. |
execute source processing | Extract data from the source |
create target table | Create the the target table and staging area if they do not already exist. |
load data to target | Load data to a staging area in the database |
merge data to target table | Merge, append or replace the data in the target table |
Last updated