Racked is no longer publishing. Thank you to everyone who read our work over the years. The archives will remain available here; for new stories, head over to Vox.com, where our staff is covering consumer culture for The Goods by Vox. You can also see what we’re up to by signing up here.

The Birchbox experiment, which began in 2010, is audacious. Birchbox charges women $10 per month for a box of samples that it receives at no or low cost from brands (though they’re now paying for more samples than they did in the past). Those samples act as Birchbox’s self-professed "marketing trojan horse" intended to inspire sales of full-size products in the Birchbox shop. Data from the more than one million subscriber profiles is mined by Birchbox and partner brands to more effectively market to both subscribers and nonsubscribers.

The Birchbox concept was once the darling of the beauty world, and so successful that it inspired the subscription box boom of recent years. Birchbox was hailed for "disrupting the cosmetics industry" by letting women who aren’t necessarily obsessed with beauty products try them at home, but the company’s aims appear to have shifted. Co-founder and CEO Katia Beauchamp told Bloomberg earlier this month that Birchbox is "not in the business of selling people samples," distancing Birchbox from the subscription service most associated with the brand.

So it shouldn't be surprising that Birchboxes aren’t delivering the best in beauty according to current and former subscribers that Racked surveyed.

Birchbox sold its boxes as curated collections, tailored to individual profiles submitted by users on the Birchbox website. While the profiles may provide Birchbox valuable data that they and their partner brands can use, the profiles don’t appear to be used in a meaningful way to determine which samples customers receive. Subscription box blogger Sarah Herrick of Bits and Boxes said that if Birchbox curates boxes based on a profile, "I must have written it in hieroglyphics."

Not only do products sent in boxes turn out to be a mismatch for profiles, certain types of products seem to be sent month after month to the same subscribers, offering fewer opportunities for discovery. "I've never felt that the boxes were curated based on my actual profile," said former subscriber Krystyn Lowe, "unless my profile screams… ‘I want all the hair products unfit for my hair and every black eyeliner ever made.’"

Makeup and hair samples pose a particular problem for Birchbox, which appears to be targeting the same white women courted by other mainstream retailers. Kim Lane said, "when they send you a makeup product [it] doesn't fit your skin tone." Martyna, a current subscriber, found the haircare customization options limited, begging, "I don't want any more volumizing sprays! I'm set for life!"

What made Birchbox worthwhile for a number of the women was the generous points-for-review store credit system. In the past, subscribers could earn a minimum of $5 per month in Birchbox store credit for reviewing samples sent in their box. A Birchbox representative confirmed via email that starting July 11th, subscribers will be limited to $5 in store credit for reviews — ever — for the life of their account. Birchbox determined "it’s no longer sustainable for us to give $5 back in points each month for a $10 subscription." Most of the women Racked spoke to doubted that their reviews were truly worth $1 in store credit each, and would have accepted reduced credit for reviews. "I understand why they needed to change the points system but this seems a bit drastic," said Amanda J. Chapman, former subscriber.

The drastic change would be easier to accept if subscribers wanted the stuff in the boxes in the first place. "The allure of Birchbox was earning $5 worth of points back each month, essentially paying only $5 a month for your box in return," said Samantha Pierce, beauty blogger at The Glitter Chic. "These points, along with a coupon code when available, made the sampling experience worthwhile." This perspective didn’t spring up in the wake of the policy change, but was rather part of the Birchbox culture for years. In early 2014, beauty blog Hello Pretty Bird explained Why Birchbox Doesn't Actually Suck, breaking down the Points system, which is "why a lot of people stay subscribed even when the boxes suck."

The end of the generous store-credit-for-review program calls into question how many shop sales Birchbox can expect once Birchbox Points no longer enable purchases of full-size products. The company routinely discloses that 50% of subscribers purchase full-size items in the online shop, but some beauty box fans said that they mostly bought products using Birchbox points and coupons — though a few did buy a beloved item at full price or check out with an undiscounted cart accidentally. Even with the lure of Birchbox Points store credit, former subscriber Allison Rummel said that she preferred to place orders for full-size products with shops that offer better shopping rewards programs. Beaux Beauty blog wrote that the change in Points policy "has effectively removed any incentive I had to make my full-sized product purchases from Birchbox."

Consumer psychologist Deborah Oyeyinka, who blogs about beauty at adoredee, said in an email, "[s]ubscription boxes are not great value for money for the customer" because the subscription format "means the customer is stuck with the product even if they want to hard pass." Yet customers tend to stick around because of the "shared shopping experience to be explored and discovered with others." With the sampling experience this reportedly lackluster, it remains to be seen how many customers will continue to discover with Birchbox. A survey — obviously unscientific — posted in the BeautyBoxes Subreddit reportedly showed that 74% of respondents were cancelling their subscriptions.

In imagining a full-size future, Birchbox forgot that trojan horses are only effective when welcomed in.

---

Watch: '90s Dark Lipstick Is Back