The Feed Doctor on Unnecessary Reundant Duplication
Long time readers are familiar with the REDUCE function that lets you “roll up” child item (or variation) data to send in a Comparison Shopping Engine feed. For example, you can write a business rule that will produce a comma-separated list of all the available sizes for a given parent item.
Somebody here at work pointed out to me that the problem is that parent items often vary by color AND size. For example, here’s an item with 4 variations:
| Size | Color |
|---|---|
| Small | Red |
| Small | Blue |
| Large | Red |
| Large | Yellow |
If I were to write a REDUCE rule to get the colors, it might look like this:
REDUCE(FUNCTION(VARS(@X,@Y),JOIN("~",@X,@Y)),$CHILDREN.COLOR,"")
and the result would look like this:
~Red~Blue~Red~Yellow
I’ve got “Red” in there twice, which looks kind of silly. What I really want is the distinct color values. So, of course, I wrote a new function.
The function is called DEDUP, and it has two parameters:
- The text to operate on
- A delimiter
First, let’s be clear on what DEDUP does NOT do. It does not remove duplicate items from your feeds. Instead, you give DEDUP some kind of delimited list (note: NOT a lookup list, but a single piece of text), and tell it what the delimiter is, and it will find the distinct items in that list, order them alphabetically, and give it back as a delimited list again. Here’s an example:
DEDUP("~Red~Blue~Red~Yellow","~")
and the output:
~Blue~Red~Yellow
Of course, we wouldn’t really put a piece of text like “~Red~Blue~Red~Yellow” right in the rule; we’d use the REDUCE rule we already wrote:
DEDUP(REDUCE(FUNCTION(VARS(@X,@Y),JOIN("~",@X,@Y)),
$CHILDREN.COLOR,""),"~")
Oh wait: I don’t want the output to start with “~”. So we need another REGEXREPLACE to rip that tilde off the beginning of the output:
REGEXREPLACE(DEDUP(REDUCE(FUNCTION(VARS(@X,@Y),
JOIN("~",@X,@Y)),$CHILDREN.COLOR,""),"~"),"^~","")
and the output:
Blue~Red~Yellow
This brings my total to only 5 posts for this whole year. Anybody want to take bets on when I’ll write another one?


