Welcome to CyberChef Charcuterie, the blog series where we slice and dice our way through the exciting world of CyberChef! In each post, I'll feature a variety tasty CyberChef recipes that I have cooked up or curated from around the web.
Just like charcuterie, these CyberChef recipes are bite-sized and easy to sample, perfect for exploring new techniques and experimenting with new flavors. From data analysis to decoding, CyberChef has something to offer for everyone, and I'm excited to share some of my favorite recipes with you. Feel free to leave a comment if you have any suggestions or recipes to share. I would be excited to include them in a future post.
Grab your keyboard and dig in!
What’s on the Menu This Week?
Elastic and Arkime are two powerful tools that are commonly used by cyber security professionals for threat hunting. Elastic is a platform that can be used to ingest, search, and analyze large volumes of data. Meanwhile, Arkime is a packet capture and analysis tool that captures and stores network traffic in real-time. One particularly useful feature of Arkime is its ability to quickly export the unique values of any column in its interface. This makes it easy to easily generate lists of IPs, domains, email addresses, and other identifiers seen on the network. Together, Elastic and Arkime can provide valuable insights into network activity and help identify potential threats.
During my work with these tools, I discovered that CyberChef could be used to quickly turn lists of values, such as IPs or domains, into query syntax that could be used in Elastic and Arkime. This has been an incredibly useful technique for me, allowing me to quickly and efficiently search through large volumes of network traffic data and identify potential threats. By using CyberChef to streamline the process of generating query syntax, I've been able to save time and improve the accuracy of my threat hunting efforts.
How To Load or Save Recipes
If you need any help managing your cookbook and would like to know how you can save or load the recipes in this post, check out this article.
Jacob Gray
Recipe #1: Convert IPs Into Query Syntax
This Recipe allows you to extract IP addresses and generate query syntax for them.
How it works
This recipe uses the "Extract IP Addresses" recipe to parse the input, sort the IPs, and remove duplicates, resulting in a clean list of IPs. It then uses a Find/Replace recipe to replace all newlines with a desired delimiter, effectively joining the IPs together. Finally, it uses another Find/Replace recipe with a Capture Group to match the entire string, and wraps the group in square brackets to complete the query syntax for use in Arkime.
Arkime Syntax
This recipe provides a simple and efficient way to generate query syntax from text containing IPs, allowing for faster and more effective threat hunting. I often use this recipe to quickly pivot on IPs from Arkime queries. However, it can also be used to extract IPs from bodies of text, such as scripts or configuration files. This makes it a versatile tool for gathering and further analyzing IPs from a variety of sources. The resulting output of this recipe can be copied and pasted into any Arkime queries that utiles an IP field, such as: ip.src == [ip, ip].
Playground Link
Recipe Code
Extract_IP_addresses(true,false,false,false,true,true)
Find_/_Replace({'option':'Regex','string':'\\n'},', ',true,false,true,false)
Find_/_Replace({'option':'Regex','string':'^(.*)$'},'[$1]',true,false,true,false)
CyberChef recipe to convert IPs into Arkime query syntax
Example
Elastic Syntax
With a slight modification, we can output query syntax that will work for Elastic. This output can be used in IP field queries such as source.ip : (ip OR ip).
To transform our output for use in Elastic, we simply change the comma in the first Find/Replace recipe to OR (note the spaces on each side). Finally, in the last Find/Replace, we change the square brackets to parenthesis.
Playground Link
Recipe Code
Comment('Example of turning a list of IPs into query syntax for Elastic')
Extract_IP_addresses(true,false,false,false,true,true)
Find_/_Replace({'option':'Regex','string':'\\n'},' OR ',true,false,true,false)
Find_/_Replace({'option':'Regex','string':'^(.*)$'},'($1)',true,false,true,false)CyberChef recipe to convert IPs into Elastic query syntax
Example
Recipe #2: Convert Lists of Strings Into Query Syntax
This Recipe allows you to create query syntax based on a list of strings. I often use this recipe when I have a list of domains or filenames to search for.
How It Works
For this recipe, we assume that the user is pasting in a list of search terms with each item on its own line. If we wanted more flexibility, we could add a Regular Expression or Extract recipe (similar to Recipe #1) to automatically identify patterns we are looking for. This one is a bit more complicated, so let's take a look at each of the recipes being used here.
Fork: This recipe allows us to split the input based on a delimiter and run any subsequent recipes against each item. It's basically a ForEach loop and can simplify certain workflows when you want to modify or process each item in a similar way. One additional useful feature is the ability to provide the merge delimiter. This allows us to easily concatenate each item with the delimiter that our desired tool uses for its query strings.
Remove whitespace: This recipe allows us to strip out different whitespace characters. We could use a Find/Replace with regex, but this recipe makes it obvious what we are doing. This recipe can be useful when we want to make sure there are no leading or trailing spaces.
Merge: This recipe is used with Fork and consolidates all the separate items from the previous Fork. Merge will also join each string with the delimiter that we specified in the Fork recipe.
Find/Replace: This Find/Replace should be familiar. It is simply taking our whole string and wrapping it in square brackets or parenthesis.
Arkime Syntax
For the Arkime flavor of this recipe, we set Fork to use a merge delimiter of , (comma followed by space) and wrap the final string in square brackets.
Playground Link
Recipe Code
Comment('Example of turning a strings into query syntax for Arkime')
Fork('\\n',', ',false)
Remove_whitespace(true,true,true,true,true,false)
Find_/_Replace({'option':'Regex','string':'^(.*)$'},'"$1"',true,false,true,false)
Merge(true)
Find_/_Replace({'option':'Regex','string':'^(.*)$'},'[$1]',true,false,true,false)CyberChef recipe to convert strings into Arkime query syntax
Example
Elastic Syntax
The Elastic Recipe is very similar. The only modifications we have to make is to the "Merge delimiter" in the Fork recipe and changing the square brackets to parenthesis in the final Find/Replace recipe.
Playground Link
Recipe Code
Comment('Example of turning strings into query syntax for Elastic')
Fork('\\n',' OR ',false)
Remove_whitespace(true,true,true,true,true,false)
Find_/_Replace({'option':'Regex','string':'^(.*)$'},'"$1"',true,false,true,false)
Merge(true)
Find_/_Replace({'option':'Regex','string':'^(.*)$'},'($1)',true,false,true,false)CyberChef recipe to convert strings into Elastic query syntax
Example
Recipe #3: Making It Modular (Bonus)
So, I might have lied a bit in the beginning when I said each of these recipes will be bite-sized. I got a bit distracted while writing this article and ended up cooking up an entree. While writing the article, I discovered the Register, Jump, and Label recipes, and learned that you can do some pretty cool stuff with them!
I decided to take some of the concepts from the previous recipes and mix them all together with some conditional logic. In this recipe, the first line of the input will be a string describing the data we want to query and the tool we want it for. So, instead of having to load in different recipes for each data type or tool, we can just specify it in the input and then paste in our data.
I'm still toying with this recipe, but I'm surprised at how well it's already working. I will probably dedicate a CyberChef Charcuterie post to it in the future, so I will refrain from going into detail about it here. I did leave comments throughout the recipe though, so take a look if you are interested and would like to know more about how it works.
Link
Recipe Code
Comment('The first line contains the process arguments. The format is PROCESS:[TYPE]-[TOOL]. Valid values are:\n\nTYPE:\n- EMAIL: This uses the Extract Email recipe\n- IP: This uses the Extract IP recipe\n- DOMAIN: This uses the Extract Domain recipie\n- STRLIST: This will process a raw list with each item on a new line. Each list item will be wrapped in double quotes.\n- IPLIST: This will process a raw list with each item on a new line. Each list item will not be wrapped in double quotes.\n\nTOOL:\n- ELASTIC: This is a OR separated search query wrapped in parenthesis\n- ARKIME: This is a comma separated search query wrapped in square brackets\n\nEXAMPLE:\n- PROCESS:EMAIL-ELASTIC\n- PROCESS:IP-ARKIME')
Comment('Here we are using Register to save the process line arguments to values that can be reused throughout the recipes.')
Register('^PROCESS:(EMAIL|IP|DOMAIN|STRLIST|IPLIST)-(ELASTIC|ARKIME)',true,false,false)
Comment('We remove the first line using Tail to prevent the process line from ending up in results when processing raw lists.')
Tail('Line feed',-1)
Comment('===BEGIN EXTRACTIONS===\nWe extract differently based on TYPE')
Comment('We can use Jump calls with the variable and match up with the TYPE specified by the process line. We can then jump to the appropriate value that is specified in the associated Label.')
Jump('EXTRACT_$R0',10)
Label('EXTRACT_EMAIL')
Extract_email_addresses(false,true,true)
Jump('JOIN_STRING_$R1',10)
Label('EXTRACT_DOMAIN')
Extract_domains(false,true,true)
Jump('JOIN_STRING_$R1',10)
Label('EXTRACT_STRLIST')
Fork('\\n','\\n',false)
Remove_whitespace(true,true,true,true,true,false)
Merge(true)
Jump('JOIN_STRING_$R1',10)
Label('EXTRACT_IPLIST')
Fork('\\n','\\n',false)
Remove_whitespace(true,true,true,true,true,false)
Merge(true)
Jump('JOIN_IP_$R1',10)
Label('EXTRACT_IP')
Extract_IP_addresses(true,false,false,false,true,true)
Jump('JOIN_IP_$R1',10)
Comment('=== END EXTRACTIONS ===\n=== BEGIN JOINS ===')
Label('JOIN_STRING_ARKIME')
Fork('\\n',', ',false)
Find_/_Replace({'option':'Regex','string':'^(.*)$'},'"$1"',true,false,true,false)
Merge(true)
Jump('WRAP_$R1',10)
Label('JOIN_STRING_ELASTIC')
Fork('\\n',' OR ',false)
Find_/_Replace({'option':'Regex','string':'^(.*)$'},'"$1"',true,false,true,false)
Merge(true)
Jump('WRAP_$R1',10)
Label('JOIN_IP_ARKIME')
Fork('\\n',', ',false)
Merge(true)
Jump('WRAP_$R1',10)
Label('JOIN_IP_ELASTIC')
Fork('\\n',' OR ',false)
Merge(true)
Jump('WRAP_$R1',10)
Label('WRAP_ARKIME')
Find_/_Replace({'option':'Regex','string':'^(.*)$'},'[$1]',true,false,true,false)
Jump('END',10)
Label('WRAP_ELASTIC')
Tail('Line feed',10)
Find_/_Replace({'option':'Regex','string':'^(.*)$'},'($1)',true,false,true,false)
Label('END')
CyberChef recipe to do it all
Example
Conclusion
That's all for this first edition of CyberChef Charcuterie! I hope you've enjoyed the samples and found some inspiration for your own recipes. If you have any suggestions for future recipes or feedback on the series, I'd love to hear from you. Please leave a comment below and let me know what you think.
If you want to stay updated on future CyberChef recipes and tips, you can subscribe to this blog. If subscribed, you will receive these posts directly to your inbox.
Whether you're a seasoned pro or just getting started with the tool, there's always something new to discover and explore with CyberChef. Bon appétit, and happy threat-hunting with CyberChef.
Thanks for reading!
