There are a couple DevTools features that make this kind of reverse-engineering much easier.
Oftentimes, the obfuscation relies on turning strings into functions. You can watch all this happen with:
`monitor(eval); monitor(Function)`
That'll log out the arguments passed to eval and the Function constructor. https://imgur.com/hMbKQZA You can also use `debug()` to breakpoint on invocation. (Yes, this works with native fns as well as user-land fns)
Then, comparing memory snapshots can help illuminate the work that happened. Take a snapshot of about:blank, then another after evaluating the ips.js payload. Select comparison. Then look through the results, especially (string). This appears to be the same "VM's strings" called out in the post: https://imgur.com/2v58VM5
Can also see heavy use of WebGL and wasm based on allocations.
Hey! My friend linked me this comment. This is super dope, I used to RE more dynamically and knowing about monitor() and debug() would have helped me a lot haha.
Also the memory snapshots, I've used that to try to debug memory leaks in the past but never thought about using it to RE. Super cool that you can also get the string that way.
You can definitely get the same results a lot faster via dynamic RE for most obfuscation methods, but what attracted me to Kasada was the challenge of writing a disassembler from a static RE perspective. I also don't think dynamic RE methods are super easy to apply here once you get into the VM logic though, as I've discovered while working on part 2.
If you want to chat with me about this more my socials are linked in the post, or you can message me on discord (umasi#3301) :)
Love these suggestions. Definitely makes dynamic analysis heaps easier. A lot of our blog posts tend to focus on static analysis but it may be worth mentioning these techniques going forward.
If you assume that every bot protection script will look at navigator.webdriver, you can just set a breakpoint before the script runs, and set a debugger statement on the property to land somewhere useful:
I have used this chrome driver project. The author probably can create something more lightweight with his knowledge. For my purposes I couldn’t find a Cloudflare bypass script that worked so I had to use this instead.
Using defineProperty can be detected with navigator.hasOwnProperty("webdriver"), which is why I suggested using it for reverse engineering rather than avoiding detection.
Blame scalpers. I work on a small time website that happens to sell limited release shoes occasionally. Its almost impossible to keep the site functional during these releases because the buying bots are both aggressive and smart. They HAMMER sites trying to get at those shoes. Its nuts.
Why not raise the price to whatever the scalpers charge or sell them on an auction instead of selling them so far below market value? If you for some reason don't want to make more money, you can donate the difference to charity. You could also do things like giveaways to previous customers or make them prizes in competitions, etc.
> That will just raise the market price, scalpers will keep scalping and overall sales will decline.
This is not true. Market price will not go up arbitrarily. Scalpers will not keep scalping if the retail price is greater than what they can sell it for, which will happen at some point.
> This is not true. Market price will not go up arbitrarily. Scalpers will not keep scalping if the retail price is greater than what they can sell it for, which will happen at some point.
How are you using such confident language on what the scalpers in this space will or will not do base on ... what looks like Econ 101 supply / demand curves? Do you have some expertise you're not including as context here?
Yeah this a fair point, I’m coming off way too confident on a complicated topic. I’d revise my comment to say:
I think this is not true, which can be seen if you make 2 basic assumptions: 1. Scalpers want to make money and 2. There is a max price people are willing to pay for shoes. Then there will be some price Nike can set which will cause scalpers to leave the market.
Why do you think sales will decline? The difference is who makes the profit. If Nike increases prices then profit shifts from the scalpers to Nike. The same number of shoes are sold for the same market prices.
Scalpers destroy markets by injecting excess capital in the marker and creating a false price floor. The issue with scalpers is they quite often get left holding the bag, and/or hold the product for a long period of time lessening the actual demand or excitement about the product in the first place.
From what I've seen they tend to move away from markets where there's not room to make a profit, eg no arbitrage. The gp is suggesting to find that price... Or make more.
I’m not sure what you’re talking about. In this case they’re not destroying anything, they’re simply performing arbitrage because of price inefficiency.
The company wants everyday people to participate in what is essentially a lottery. It's the psychology of them being priced at that level, giving them away wouldn't have the SE effect.
Similar with music tickets. Maybe musicians want everyday people to afford going to their shows, even if the market demand would price out many of them.
There's a 2013 NPR interview with Kid Rock about the comparable problem of concert ticket scalping prices. His solution: do more shows in popular cities to satisfy demand; reserve a few premium seats and price them at scalping market prices so he's making a good overall profit; and (not applicable to shoes) bind the ticket to the buyer and require ID at the door.
There's a variety of second-order consequences that aren't addressed by simply trying to price out scalpers/capture excess willingness to pay.
unethical_ban lists some of them.
There are others. For example, it is psychologically quite different to have scalpers say change prices every day or even every hour than to have a large company change prices every day or every hour for the same product. You can have considerable customer ire directed at the company in the latter case.
In a similar vein to what unethical_ban said about lotteries for everyday people, the hype that a company builds with products at "everyday prices" just in limited quantities is different and targets a different audience than products that are priced at luxury prices, even if the product on a secondary market commands the luxury prices.
Simply changing the price to try to deal with scalpers is an extraordinarily blunt tool that can run counter to many other priorities a company might have.
Part of if luxury good appeal is that that proof of work creates additional value. For example if you had be the first to solve a complex puzzle, wait in line for 24hrs or even have the fastest bot to get the item. That creates hype and adds to brand value.
Nike then cashes out that value by making partner stores Buy their shitty products to get a chance to score the good stuff
Why don’t you just use a different distribution mechanism that prioritizes existing customers? Real buyers can’t beat the scalpers anyway. There is no bot protection that works. Even if it did scalpers can checkout faster and make more attempts than your regular users using manual methods.
Absolutely, but it misses the point. The bots are a feature not a bug. They create scarcity and improve brand value but only to a point. You can’t have them be 100% of the market but 75% is probably ok
Cool write up, reminds me of a .net app I had to RE a few years ago which used a custom bytecode/VM for sensitive functions, such as some string crypt/obfuscate operations which I was trying to grok. Giant pain in the ass.
…until I saw that they re-implemented the same operations in some schema migration code without adding the magic ”obfuscate-me” annotations
I ran into the same thing reverse engineering a piece of exercise equipment
The routines for communicating with sensors were heavily obfuscated in the main application... but the factory sensor test application had been left installed and was completely clear.
I came here to say exactly the same. Usually I can enjoy idiosyncratic fonts, but it's driving me crazy trying to read this.
It's the form of "f" used in italics, only un-italicized.
I think part of my brain keeps thinking it's the italic "f" in math notation "f(x)" and then gets confused to find it as part of a regular non-italic word.
Or it might be the fact that our brain relies heavily on word shape for fast reading, which is heavily dependent on the presence or absence of ascenders and descenders in lowercase, so adding a descender on a letter than shouldn't have one totally breaks our natural pattern recognition.
In any case, I've never come across it in any typeface before in my whole life. Now I'm starting to see why. :)
Removing the top pixels from characters greatly reduces readability, which is not as significant if you remove the bottom ones. Then why does this happen? Any ideas?
We don’t really read by looking at individual letters. At least, once we become proficient readers. Instead we read words as shapes. If the shape is too far off from the pattern stored in our head, then we have to stop and actually look at the letters to read it.
> One technique to protect against these attacks is browser fingerprinting. This method works by collecting data about a user's browser, which is then used to create a unique fingerprint for differentiating between genuine users and bots.
This seems like it can't work. 50% of users are browsing from an iPhone. Every iPhone of the same model has the same fingerprint except for time of day and language preference. So for every time zone there are literally hundreds of thousands of devices that will have the same fingerprint.
The EFF tool is provably false. For one, it doesn't get enough traffic so it's not representative in any way shape or form of a website that would use fingerprinting.
But it also doesn't pass the sniff test. It tells me I'm unique. One of 185k. Let's break down that number. I'm in the PST time zone. There ~40 million people in the PST time zone. Divided by 185k is 216. They're basically claiming there are 216 iPhone 13 Pro in all of California + Oregon + Washington + British Columbia.
Bullshit!
It's basically fake news. The EFF should know better than to exaggerate with hyperbole. It might be true that almost no one visits https://coveryourtracks.eff.org/, but site that actually gets traffic is also a sight where it'd be closer to unique in 1 of 10.
That site doesn't generally have sufficient recent data, it's essentially saying device fingerprint is rare compared with data from older software versions.
EFF don’t say they know who you are, rather how unique your device appears. This is a problem because if you appear very unique, only if one site knows your identity, everyone can potentially know it.
When you generate a report they tell you which bits of your device were unique. For an iPhone running the latest iOS there is nothing in that list that leaks a significant number of bits.
My point is they don’t claim any of the bits are unique (all of the individual things are very common) and still someone end up at an answer that is “unique”.
The goal of fingerprinting in this case is not to uniquely identify users, but merely to differentiate real human users using an actual iPhone from bots that use an emulated iPhone.
This fingerprint suffices for that. They use small differences like the size of audio buffers, the exact capabilities of the GPU, etc.
Emulate any one of those slightly wrong and the fingerprint will differ and your bot will be revealed.
> They use small differences like the size of audio buffers.
Fun fact, on my laptop I can hear websites who use audio APIs for fingerprinting because it causes the audio subsystem to wake up and the speakers make a small pop sound.
They may be referring to JA3 [0] which can be used to fingerprint the connection by examining the order of the ciphers sent by client to server during the TLS handshake. It isn't useful to detect that the user is browsing on an iphone, but rather that the useragent says the user is using an iphone but the JA3 gives it away as something else.
It can be defeated by the client sending a specific list of ciphers in a specific order during the TLS handshake but in practice this can be difficult to do as it's typically implemented at a low level. Alternatively, creating a browser extension can also be used to defeat it as the request is running through the browser's code in that case.
I am not sure why can't you use other features of the traffic that such website gets. Isn't there any better way of finding features that are great signals that a request is a bot?
We used to work on a project where the traffic was inserted into a graph like data structure and we connected the known bot traffic nodes and calculated the fraud score based on that. It worked very well.
That would happen server side and be invisible to you reading the js. The cam easily be doing this and we'd never know unless Nike told us, which they will not for obvious reasons. But a lot of people just buy the shoes.
There's a hard tradeoff here between how long a user on a low-end device is willing to wait to purchase something and how quickly a bot with powerful hardware can go through your proof of work.
AFAIK those scripts are more to prove that the client has a functioning javascript environment rather than proving that they can compute 5s of computation or whatever.
Why does Nike bother? Their home page just has some pictures of stuff they sell. It doesn't do much. It leads to apparel ordering pages. Those have a column down the left for selecting some options, and pictures of products. Just like everybody else that sells clothing on line.
People seem to lose their minds when new sneakers become available, which instantly have a huge resale value. So the anti-bot measures are probably to try and counteract sniping of said sneakers.
The most powerful anti bot measure would be to make an order of magnitude more shoes. Then everyone who wants a pair can have one, Nike gets 10x the revenue, no reason for bots to game the system, no need to pay VM experts to design anti bot obfuscation.
I guess the halo effect of releasing unobtainable shoes is worth more than all that? If so, why does it matter (to Nike) if bots buy all of them?
People desire the shoes due to the scarcity. Producing more supply would lead to a commodity and less popular shoe/brand over time. No one talks about shaq shoes from Walmart because they are using the model you are suggesting
People desire the Shaqs too - apparently 100M+ have been sold. At 5k to 20k units each time, hyped release Nikes will never ever add up to that type of volume and can never contribute meaningfully to a $200B company's bottom line. They have to be viewed as an advertisement that somehow adds prestige to the shoes they actually make and sell at high volumes. It's probably a much less important advertisement than the big name athletes they sponsor too - I bet they could discontinue the hype releases and not lose much market share.
You have absolutely hit the nail on the head; here.
I think - Ye aside - Yeezy is a great example of how there can be a balance between this like insane run of only 1000 shoes (which only sell for like $250-350 USD retail anyway…so - yeah - where’s the profit again?) - to mass market ‘Wal-Mary’ level availability.
The only people who benefit from limited releases are scalpers. Period. There’s no way the shoe company makes that much profit off a limited release because the limited releases are wildly somehow still usually at ‘regular’ prices.
A solid pair of YEEZY 350’s is not that hard to come by for any even half-committed sneakerhead, I’ve got 3 pairs and my girlfriend has two. We hardly tried. I won the lottery twice myself, and only ever entered 6 times.
It’s really a shame what happened with Ye, because despite all the drama; the Yeezy fashion project was actually an interesting one that had some real potential to find an actual, honest-to-God balance between artificial scarcity and actually selling the goddamn product.
So Nike arbitrarily limit the availability of a shoe. People for 'reasons' want to buy the shoe because of limited availability. People are willing to pay more than what Nike is charging. 3rd parties come in to capture that wasted value. Nike wants to stop those 3rd parties for 'reasons'.
There seems to be a lot wrong with this situation. A group using free market, capitalist tools in a free market capitalist country seems the least wrong part.
> 3rd parties come in to capture that wasted value.
That value would otherwise have gone to the people. If all value is captured in this way, the marginal benefit of making a purchase becomes very, very low, and the stuff we actually care about (standards of living, happiness, liberty) become decoupled with economic metrics.
This is one of the situations where lotteries are fairer than auctions.
Ok.
"3rd parties come in to capture the value that Nike otherwise left on the table"
I disagree that all value would be gone though. People value things differently, the 'scalpers' are a mechanism to get the goods to those who value it most highly. I personally don't value these trainers for what Nike is asking, so I don't buy them. Should Nike be required to sell them to me for what I value them at? Or should people be able to set their own price, and others buy if that price is worth it to them? If you actually cared about happiness, liberty, etc, I don't see how a pair of trainers, expensive of otherwise is going to change that, but if it does become an issue, just don't by the sodding trainers then!
I don't see why a lottery is 'fairer'. the shortage is entirely synthetic, created by Nike. I'm not big on adding morality into business transactions, but all this stems directly from Nike's decision to create scarcity. The 'fairest' thing would be for Nike to satisfy demand. 'scalpers' are a just symptom of that.
I don't get why 'scalpers' are only deemed to be a problem when the scarcity in question is synthetic. If there's a corn shortage are farmers scalping? Are oil traders scalping when the price goes up? What about when the price of $stock goes up? Are the sellers scalpers? This is how free markets work. You could at least make a moral case for the price of corn being socially bad, you can't make that case for trainers or gig tickets or whatever.
The other obvious solution is for another company to come in and solve the supply constraint that nike created. Although that too is also apparently wrong. If I bought some 'Niko' trainers from the local market, who is being harmed here? I'm not being misled, Nike aren't losing sales, so why?
1. "those who value it most highly" is a dream of economists that has never ever been true. Personal value is relative to wealth and income. I might value something at one full month's salary, but an average CEO could outbid me with a few work days at most.
2. "prices go up when demand is high" makes sense to incentivize more production and reward those that produce thing we need the most. A farmer getting more for their corn during a shortage makes sense, because we'd be hungry if it weren't for them.
A scalper, on the other hand, does not create any value. Neither does a stock trader when they buy low and sell high for that matter.
Even artificial scarcity of luxury items can be interpreted as creating some value - owning a limited edition shoe that they can brag about to their friends is valuable to some people. But buying something before others get to it, just to immediately sell it for a much higher price is just exploitation.
Re 1, have you got a better plan? There's communism, are you suggesting that?
>2. "prices go up when demand is high" makes sense to incentivize more production and reward those that produce thing we need the most
Precisely, so why isn't Nike producing more? That is the issue. Again, scalpers are a symptom.
>A farmer getting more for their corn during a shortage makes sense, because we'd be hungry if it weren't for them.
One doesn't follow the other.
If a farmer gets £1 a tonne one year, why does it mean that them getting £10 a tonne the next mean that people won't go hungry. If there's a corn shortage, that implies there isn't enough, so people are going hungry. Further this seems to verging on a moral argument. If we go hungry if it isn't for farmers then that implies that the state should be stepping in to control their excesses, setting the price at £1 a tonne.
>Even artificial scarcity of luxury items can be interpreted as creating some value
And so can buying a good and selling it on for a higher price. Why does one person have the right to brag about owning a shoe, and not another, who just so happens to be willing to pay a higher price? Why is artificial scarcity acceptable, but creating a functional open market out of that not?
And a stock trader does create value. If a person can't buy stock and sell it on for a higher price, that stock is worth less, which means during an IPO a company can raise less money. It's not even clear to me if you could have PLCs at all, because eventually everyone dies, what happens to a company who is owned by a dead person(s) does it get taken over by the state?
If we were talking about actual necessities then I'd agree with you, although my solution would be to have Nike produce more, not hold people to ransom. But for trainers, a specific brand and model, let the free market function, it's the best we've got.
> Why does one person have the right to brag about owning a shoe, and not another, who just so happens to be willing to pay a higher price? Why is artificial scarcity acceptable, but creating a functional open market out of that not?
It's a luxury for poor people. Making it a market turns it into a luxury for rich people
I'm not American, I'm European and somewhat left of centre at that, as you may have detected from my use of the word trainers. the parent was suggesting moving away from a system where those who are willing to pay most, get a good. What other options are there? Socialism? But to me a socialist state that starts controlling footwear is communist anyway.
What dogma are you proposing I have? I know capitalism has its problems, I know communism has its problems, as does socialism and all the other isms. If capitalism is good at anything though, it's good at making trainers available. If youre saying it's unsuitable for that, then it's time to find something else.
It's ok saying the current system is bad, but if you're proposing to get rid of it, you need a replacement and as Churchill may have said, free market capitalism is the worst economic system, except for all the other economic systems.
So why don't you debate my position, rather than my assumed nationality, dogma and cultural baggage?
There's a big resale market, sometimes shoes that cost $250 go for $10K resale. I used to develop bots for this as well and documented some of the experience: https://juuso.dev/northern-nike-nabob.html
I strongly dislike that you engaged in the shoe scalping arbitrage business, but at the same time really appreciate you sharing this information. Excellent writing covering a fascinating adventure!
Question: With the Twitter RSVPs, do you think Nike implemented automation to automatically give you the code when you responded with the correct code in a fraction of a second? If so it implies that on some level Nike was fine what you were doing.
Eventually Nike made changes into how they were giving out the codes, so they tried at least. E.g., 0 follower accounts could not gain a code. We switched to using accounts of my contact and his friends to solve this issue. I think the idea of adding a timer for artificial latency was also discussed, but I can't recall if we ever implemented it.
In general we thought that Nike was complicit to an extent. Especially with the on-site checkout and mining of product IDs, they could have cancelled orders done before they shared the link in public. I think on their end, it was a balance of guaranteeing a low supply while ensuring that normal customers did not feel like they were being screwed _all the time_. Yet, even nowadays with the SNKRS app some continue to call it a rigged ballot.
In hindsight, Nike is still positioned somehow uniquely in the sneaker community: it's not guaranteed that money alone will get you a pair. Compare this to luxury brands like Balenciaga, which retail sneakers with a price tag of $500-1000. With the latter everyone knows you have handed out a chunk of cash, but it's not the same thing as the story that you could have waited in line or woke up in the morning to enter a raffle, or been smart enough and early on to pay reseller prices to get a pair before stock ran out which you knew would become a coveted pair over time.
In the past, people needed to queue to get their hands on the fresh stuff and there was a clerk that cloud verify your identity. Of course folks could send in their friends or use fake documents, yet the degree of the procedure was limited to humans in their flesh.
e-commerce sales open the gates to the shop at the same time for everyone. This incentivises massive parallelization of the purchases across accounts, IPs, and identities. Already back in 2017 [1] it's become known to the industry that bots are sniping the purchases faster than humans. Now, we're at a stage where retailers are adjusting their T&Cs to address the resell market [2]. StockX was valued at 2.8B in Dec 2020 [3] and they have to get stock somewhere, right?
The bot industry [4] also developed from simple scripts to SaaS offerings with multi-thousand $ subscription prices just to get access to a bot that's proven to work. The user has to provide a list of proxies that will be used to execute the scripting as these are not provided out of the box (and are hard to get). There are numerous guides on YT and advertising on Twitter that show how to use the bots and how they've been used for purchases. And of course, scarcity is not just for streetwear, but for any type of highly sought for products [5].
the "scalper problem" is strikingly similar to many cryptographic problems - the "sybil problem" in token distribution ("distribute 25 bitcoin every 10 minutes to a random participant") is obviously similar to distributing scarce items which are scalpable (sold below market clearance price).
"We'll just have a free-for-all (random distribution)"? Oh ok I'll register 2 million network participants (or IPs, or email addresses, or browser sessions, or privacy CC numbers, etc). Without some kind of outside oracle proving user uniqueness (and many many oracles that people think of, like CC number or billing address, are not actually unique!) it is extremely extremely difficult to construct this system.
NVIDIA distributing some 4090s via GeForce Experience is another interesting example of a hardware oracle, the telemetry is a fairly strong signal of hardware authenticity (as proven by Overwatch's fairly effective hardware bans).
I once met a guy who had a stack of 50 sim cards used for bot buying sneakers and playstations. Guy didn't work in tech but had learned javascript and selenium for scalping. You can also ship a lot of things to the same address by just altering it slightly like setting the unit/number to 1B or something when there is no 1B, the delivery people will get it to the right place but the unique check will think it's a different address.
He had apparently made enough money doing this to pull himself out of poverty and get a decent life going.
After seeing the URL, I was a bit disappointed they do not let sneaker heads make drawings of shoes as a CAPTCHA. Don't require any artistic skill, just some creative effort - worst case, the bot makers will make some interesting contributions to AI art.
There's not much you can "do" purely server-side that isn't easily bypassed by bots. That's how the arms race kicked off.
websites rate limit IPs -> bot developers start using proxies
websites block datacenter proxies -> bots use residential proxies
websites add captchas -> bots start solving them
websites inspect nuanced details of TLS fingerprints and header ordering -> bots start faking them
websites add browser fingerprinting -> bots reverse engineer it and start faking it
And so on. One thing these websites could do is place a lower limit on the total checkout time. Anyone who completes their checkout in less than 2-15 seconds (depending on the website) is quite obviously a bot.
Which they never do and even if they did they would get botted in using more volume. Only viable method I knew was sms verification, but that didn’t seem to be implemented correctly by anyone.
Oftentimes, the obfuscation relies on turning strings into functions. You can watch all this happen with:
`monitor(eval); monitor(Function)`
That'll log out the arguments passed to eval and the Function constructor. https://imgur.com/hMbKQZA You can also use `debug()` to breakpoint on invocation. (Yes, this works with native fns as well as user-land fns)
Then, comparing memory snapshots can help illuminate the work that happened. Take a snapshot of about:blank, then another after evaluating the ips.js payload. Select comparison. Then look through the results, especially (string). This appears to be the same "VM's strings" called out in the post: https://imgur.com/2v58VM5
Can also see heavy use of WebGL and wasm based on allocations.