Complex data challenges often requires multiple iterations or workflows to solve. Trifacta can help you organize complex workflows using a Plan.
What is a Plan?
A Plan is a sequence of tasks that are executed based on a trigger. Plans can be used to automate the execution of multiple tasks, ie. flows, using the outputs generated from one in the next. As the name suggests, plans allows the user to map out how they want the tasks to be executed.
A Plan consists of a flowchart of tasks linked to each other based on sequence and when the plan is run, the tasks get executed based on the flowchart.
Plans can run on-demand or on schedule.
You can cherry-pick which outputs to produce, or produce all.
You can override any flow level parameters during execution of plan
Create a Plan
Plans allow you to orchestrate data pipelines. The sequence defines how the tasks should execute.
1. To create a Plan, you must first identify the tasks that you wish to execute and ensure that you have:
Access to all the flows that you wish to include in your flows, including the recipes and datasets within it.
The flows have at least one output
2. Go to Plans page by clicking on Plans icon in the left nav bar
3. In the Plans Page, click Create Plan
4. Provide a Plan name and Description
5. This opens up the Plan view where you can create and configure your data pipeline and execution settings.
Changes in the Plan View are automatically saved. See Plan View Page for more details.
6. In the Plan View, click the Plus (+) icon at the bottom of your plan to add a new task.
7. Specify the type of task (Flow Task / Http task) in the right side panel. To run a Trifacta flow, choose Flow Task. For more information on Http task, see the following section.
8. Choose the outputs and click Create Task.
9. Repeat steps 6-9 to add additional tasks as needed
Adding an Http Task
Http task allow you to integrate Trifacta with other applications by sending a request via HTTP, ie. a Webhook. To add Http tasks:
Select Http task option from the right panel when adding a new task
Enter the URL , Method, Headers and Request Body as needed.
Test the call by clicking on Test
Save to add this task to your flow.
You can insert a new task in between the existing sequence by clicking on the Plus (+) symbol in between two existing tasks and selecting "Add a node"
Adding Parallel Nodes
You can add Parallel tasks by clicking on the Plus (+) symbol above an existing task and selecting "Add a Parallel Node"
Task Run Status Rules
You can configure the execution of tasks in your plans as per the status of previous task run results. For example-
Run Task2 only if Task1 is successful.
On Failure of Task1, run the cleanup task of deleting records and sending an email
With this powerful capability, you can decide on what status of the previous task (execution/success/failure), its dependent task needs to run.
As you add sequential tasks, you will see the status rule option appear on the connector between the two tasks. By default it is set to 'On Success'
You can click on it to see other options available to set as Status Rules.
Select the correct option as applicable to your workflow.
Execute the Plan
Click on Run Now button on top to execute the Plan
Once the run is complete, it shows up in the Plan status in Plans Page
Go to Jobs Page and see Plan Runs in Job History.
Expand the Plan you have executed to see details of that run.
You can monitor the progress of your plan runs and review all previous plan runs. See Plan Runs Page.
Schedule a Plan
To execute a Plan repeatedly at regular intervals, you can create a schedule for it.
Open the Plan
Use the More(...) icon on top right and select Schedule option
Enter the details of the schedule like day, time, frequency, time zone etc.
You an enter Parameter values to override your Flow Parameters also
The Plan will now execute as per the schedule created.
You can also schedule an individual flow to automate your results.
Note- While a plan is in development, you may wish to disable its schedule, which prevents execution according to the schedule. You can still run test executions using the Run Now button.
To learn more about Plans, read this detailed documentation guide
Learn more about the new features in Plan View including Parameters and Better Monitoring
Also see the following related articles for further details-