Announcing the Splunk SPL to SPL2 Converter

Whoa, Splunk Ingest Processor is now Generally Available! If you haven't already, you might be getting started with building your first pipelines Ingest Processor, or perhaps you've already been exploring Splunk Edge Processor. You've logged into the user interface and poked around - maybe you've even started sending in data over HEC, forwarders, or Syslog! But now you're stuck. How do you write your first pipeline? Sure, you know SPL, but SPL2? In a streaming context? How do you even get started?

Introducing Splunk's SPL to SPL2 converter, now available for Splunk Data Management's Edge Processor and Ingest Processor (herein known collectively as "pipeline builders!") Simply copy and paste, or hand-type, SPL in your pipeline, and Splunk will convert it to SPL2. It's a fantastic tool for getting up & running with your first few edge or ingest processor pipelines, allowing you to learn SPL2 along the way - you can even copy & paste existing SPL searches from Splunk Cloud Platform or Splunk Enterprise to turn search-time analytics into pre-ingest transformations! (So goes the proverb, "with great power comes great responsibility.")

Let's learn more.

How to Convert SPL to SPL2

Converting SPL to SPL2 is simple. Let's start with a basic, empty Edge Processor pipeline:

In honor of Splunk's recent acquisition by Cisco, let's say that this is Cisco ASA data, and we want to extract the message number field from _raw, filter out a specific message number, and set the vendor name as "Cisco". Well, how would you write that search in SPL? We already know how to do that!

| rex field=_raw "/(%ASA|%FTD)-\d+-(?P\d+)/" | search message-number != 43003 | eval vendor_name = "Cisco" | fields - message-number

Now, how do you use this SPL in an Edge Processor pipeline? You have two ways:

1. Use the spl1 command directly
2. Use backticks (`like this`) - this is shorthand for the spl1 command

All you have to do now is write your SPL query in the spl1 command, or use backticks. If your query can convert to SPL2, clicking on your query should result in a lightbulb appearing on the left side. Alternatively, you can hover over the query and click "Quick Fix". Click on "Convert SPL1 to SPL2" and voila!

You'll notice that one of the minor differences in SPL and SPL2, the fact that field names with non-alphanumeric characters (or the underscore _ character) must be 'single-quoted', is automatically addressed - the converter added single quotes around message-number.

But wait, what about the search command? According to the command support matrix, Edge Processor doesn't (yet) support it. No worries - the converter will also try to convert search clauses as well, to become the where command (with exceptions):

What to Expect When Converting

When converting from SPL to SPL2, keep a few things in mind.

1. This converter is a UI tool only, to help you build your SPL2 pipelines. The pipeline builders will not be able to compile any pipelines that have the spl1 command or `backticks` remaining in the pipeline, as indicated by the SPL2 command compatibility matrix. That means that if you try to convert your SPL into SPL2 but the conversion is unsuccessful, or spl1 or `backticks` remain, you will receive the error "Command spl1 is not supported in profile 'edgeProcessor'." You will have to rewrite your pipeline manually.
2. Just because a command converts to SPL2 does not mean that it's supported in the pipeline builders! That's because many SPL2 commands, such as timechart, are supported by splunkd, the search runtime (which you can access using SPL2 by clicking the "Search" tab on the left side of the Data Management homepage), even if not (yet) supported by Edge Processor or Ingest Processor. You'll see that when you try to run the pipeline, and get an error similar to "Command 'stats' is not supported in profile 'edgeProcessor'."
3. As much as possible, the converter tries to be profile-specific. That means it may handle conversions in the pipeline builders slightly differently than search conversions targeting splunkd. For example: the search SPL2 command is not supported in Edge Processor or Ingest Processor, because search is reserved for data at rest. When you convert the SPL search command in a streaming pipeline, the converter tries to understand the context and will often convert it to the SPL2 where command or clause, as shown in the second GIF above.
4. You can convert multiple sections of your pipelines individually without converting other sections, by mixing SPL and SPL2. For instance, your pipeline can follow this pattern:
   
   

   $pipeline = from $source |  |  |  | into $destination
   

   
   and the SPL to SPL2 converter will apply to all sections of SPL-only syntax. Like before, all SPL must be converted to SPL2 before trying to run your pipeline.

Because SPL's search and SPL2's where are not always semantically identical, and for best practices overall, it's highly recommended that you test the pipeline first before deciding that the converted SPL2 reflects the original intent of the SPL.

Start Converting!

Go on, nothing's stopping you from trying it out yourself - start up your backticks & let's get ready to rumble!

For a full explanation of the SPL to SPL2 converter's capabilities and limitations, visit our documentation.

Want to see more kinds of supported SPL to SPL2 conversions? File an idea for Edge Processor on the Splunk Ideas portal!

Want to chat about SPL to SPL2 conversions in general? Find us in the #spl2 channel in the Splunk user-groups Slack!

Public link

Please use the above public link if you want to share this noodl on another website.