Pipelines are one of the features that sets Search.io apart from other search solutions on the market today. On a high-level, there are two main types of pipelines:
- Indexing, or record, pipelines which allow you to modify and transform data as it is loaded
- Query pipelines that define how queries are executed
We’ve supported live relevance editing in the query pipelines for a while — change the algorithm and see what results look like live, see below. This has been a game changer for customers coming from other popular products today that actually duplicate the entire index to make it fast and charge you double for the privilege!
The new real-time pipeline transformation preview feature extends this amazing functionality to indexing pipelines. It provides an elegant way for technical users to debug and adjust their pipeline steps by showing exactly how each transformation works and what the end result will look like in the index.
A new preview pane displays pipelines side-by-side with the Request and Response.
- The Request pane shows your raw data from a record that you would send into the pipeline.
- The Results pane beneath it shows how the sample data will be transformed by the pipeline.
For example, you might have a pipeline step to remove HTML markup from your index, strip symbols, find and replace, perform mathematical calculations, and much more. You can test data in the Request pane and how it’s reflected in the index in the Response pane. In the video above, we show an example of stripping out symbols from HTML.
Because pipelines also allow all config to be represented as code, you can also comment out steps to see what happens, do diffs with previous versions and all the other nice things you would do with the rest of your code!
The new indexing pipeline transformation preview offers simple, yet powerful, features that give customers more flexibility and control over their data.