Scheduling With Cron Expressions in Integrate.io

One of the most requested features in a data integration tool is greater flexibility around the scheduling of packages and workflows. With Integrate.io, this can be achieved through the use of our Cron Expression scheduling feature.

What Are Cron Expressions?

Cron is a software utility that enables Unix-based operation systems, such as Linux, to use a job scheduler. You can create cron jobs, which execute a script or command at a time of your choosing. Cron has broad applications for tasks that need time-based automation.

The cron job gets its instructions from a file known as a cron table, or crontab. You can set up crontabs for individual users or for the entire system, depending on what you’re trying to accomplish with it. The file structure is set up with a single job on each line, created out of cron expressions.

A cron expression typically contains up to five fields that define the time for that process, along with white space. These fields are:

• Minute field
• Hour field
• Day of the month field
• Month field
• Day of the week field

These five fields are self-explanatory. For example, day-of-week allows you to designate the specific day to execute the package. In some types of cron deployments, three more fields are available:

• Account Username
• Year
• Seconds

You only use the account username field when you set up system crontabs. This field is the username on that system responsible for running the cron job. It must have the appropriate permissions to work properly. Some cron versions also add a year and seconds field into the mix for greater control over scheduling.

Cron expressions default to using the system’s time zone, but you may need to adjust this if you have a distributed team in different time zones or a similar situation. When you’re using Integrate.io’s cron expression feature, the designated time zone is UTC.

Each field has a set of allowed values, which are the only values that are valid to use for that particular field.

Special Characters in Cron Expressions

Some of the allowed values in a cron expression include special characters. They’re defined as follows:

• Asterisk: That field contains all of the allowed values. 
• Question mark: If you don’t require a specific value for a field, you can use a question mark to set up the right crontrigger.
• Comma: You can use multiple specific values in a field by separating them with commas.
• Dash: Set up a value range with a dash.
• Slash: You skip the specified values.
• Pound sign: Create a cron trigger for the nth day of the month with this special character.
• L: Use this for choosing the last day-of-month or day-of-week.
• W: You can select a weekday in the day-of-month field.

Examples of Cron Expressions in Integrate.io

Cron expressions allow much more flexibility in your job scheduling by supporting irregular intervals, something not possible with our interval-based scheduler. Here are some examples:

0 8 * * * Every day at 8 am UTC

0 8 * * MON Every Monday at 8 am UTC

0 8 1 * * Every 1st of the month at 8 am UTC

0 0 L * ? Last day of the month

0 0 15 * ? 15th day of every month

0 0 ? * FRI Last Friday of every month

Who Benefits From Using Cron Expressions?

Integrate.io’s interval scheduling tool is designed for fixed intervals, which you can configure for minutes, hours, days, weeks, and months. This feature works best for packages that operate on a highly predictable timeframe and often cover your most common use cases.

The addition of cron expressions empowers you with fine-tuned control over your package execution schedule in your data pipeline. You can fill the gaps in-between your routine intervals, configure a schedule for temporary and ad hoc requirements, and adjust the cron jobs as needed to best fit your requirements. You can schedule tasks around your active and inactive hours, ensure that important packages get executed consistently, and remove the need for manual input from your staff.

Using Cron Expressions with Integrate.io

This feature is now available on all Integrate.io accounts. You can easily set up a cron expression by clicking the time icon in the scheduling tool, which gives you presets to select from. If none of those work, you can write your own cron expression or use a cron generator for the correct syntax.

When you’re setting up these cron jobs, consider how long it takes for the package execution. By default, if a previous job on the same schedule is still running, it won’t execute. You can adjust the increments of your cron expression or choose the “Allow concurrent schedule executions” option if you want to avoid this situation.

For further reading on this new feature, check out our scheduling documentation for a complete tutorial.

Ready to try Integrate.io for yourself?  Schedule an intro call with our support team to find out how you can get a 14-day risk-free trial.