I'm the engineer he's referring to. I still get a laugh seeing Justin's amazement when we told him her name. To answer a few questions...
- This isn't the actual photo but looks similar, except that she was alone. Justin was afraid of reposting the same photo after the results of handing over a photo last time :)
- There was no useful exif data in the image.
- We knew her high school and approximate age in Austin, but were unable to find any yearbooks / class rosters online. (Idea not pursued: try to acquire a yearbook offline).
- Facebook Graph Search and LinkedIn with the criteria we knew didn't help.
- Google Image search didn't help.
> "I'm sure there's a lot of information missing from this story"
Quite true.
Ultimately we had several methods going at once to try to figure it out.
The one that came back with results first relies on some information Justin still doesn't know we had (though he would still be impressed), but other approaches we also expected to work would be possible with only what's in the story, and haven't yet been mentioned here.
* Everyone was at the original bar at the same time - someone else had photos from the night that had her and other people in them, and those people (her friends) could be found online.
* The bar had some kind of "sign-up for x" list, or some kind of contest, or some kind of photos of the night, which were helpful.
* You searched twitter/facebook/whatever for a girl posting about a boy she just met at bar X, blah blah
* We know she was going to some kind of event - you guys figured out what that was, and found a guest list or something.
* You guys did a some wi-fi sniffing and read a few emails. He did say he emailed you the photo, so you know which account to listen for.
If it were me, the first thing I would do knowing her high school is create a Facebook account and upload her photo (with her face) and see who Facebook suggests I tag in it.
I'm not sure what the rules are concerning tagging suggestions, but I have noticed that it suggests that I tag people I know in other people's photos so you KNOW they recognize everybody in every photo - the trick is meeting the right conditions for it to allow it to share that recognition match with you based on her privacy settings I think...
So in the other post, you said you inadverdently realized you had a second photo of the target (which presumably contained key identifying info)...so, are you saying here that that second photo didn't come from investigating the event that she was at (i.e., if it had a meetup/lanyrd page, looking at all the attendee photos and seeing what looked like her?)
The second photo that we later realized we had was one a teammate had taken himself and didn't have anyone else in it. I think it was doable with just 1 photo, and I'm not ready to share with Justin how the 2nd photo helped. :)
An article of clothing or accessory of some kind from a conference she attended that helped you look up attendees and identify her, or some other identifying club/service that contains profile pictures?
In a single picture where she was a random bystander? Unlikely that you'd be able to read anything that had her name on it, unless it was an actual name tag. Even then, people don't generally stand very still for photos that are taken of other people.
You looked on Instagram and other for pictures taken at the bar, and you found one either taken by the woman in question or one of her friends with the mystery woman tagged.
Or maybe you knew that she wanted him to meet her at an event and you found a roster for that event- or ie if it was a real estate convention you would have a pretty good idea that she was a realtor and could look at websites for realtors in Austin.
Or maybe you checked his recent facebook friends, looked at his recent twitter followers, and everyone who liked/commented on his recent photos/posts on Instagram and whatever other social networks he's on- as well as the photos/posts he liked/commented on.
I think you took a (relatively) low-tech approach to this.
It was this line -
"On Thursday night, after exchanging texts all day, around 8:00pm local time the girl sent me a picture of her in order to convince me to meet her at an event she was going to."
Find the event Thursday night at 8:00pm in Austin and start tracking down from there.
...which, possibly, would include a picture of that person either in the same outfit or include a profile picture of that person commenting or liking the event.
Your facebook link contains a list of 73 persons going to the event, so maybe one of them had a picture that matched the one the team had of the girl...
post pic on craiglist/similar in Austin area. Say you found wallet/purse/camera/etc with her picture. Anyone that knows her please call/email w/ Name/Address for you to send back. Use highschool for validation when contacted. Might have some error.
This is the closest I've heard to one of the ideas that we expected to work, so I'll share.
- Create an ad on Facebook with her picture.
- Target people that went to her high school within 4 years of the range of years we thought she graduated within. Was only a few hundred people so the cost of running the ad would likely only be a few bucks.
- The ad copy / landing page just needed to be convincing that we weren't stalking her. We went with the "help us win a bet" approach but the "lost camera" approach would have been good also.
Ultimately we got the answer sooner from another approach after we found out that we had another picture of the girl that Justin didn't know we had. So we canceled the ad, but I think this would have worked if given enough time.
You had another picture that showed where she worked? Used LinkedIn and company's webpage (Some company list their employee) to narrow it down to a few people. Or send a message to a coworker on linkedin whatever looking for information about that girl and they gave it to you? I'm curious :P
This method likely would have worked by itself. The method that worked sooner wasn't just "another photo" found online of her friends as grecy mentioned. It was a photo we had taken ourselves, and it didn't have anyone else in it. But I'm not ready to tell Justin more details about it.
This strikes me as odd, albeit a clue. You had a photo of a girl you'd presumably never met. A photo you'd taken, that you didn't know was her at first.
They went to the same event, took some photos. And in checking their photos, see that the same girl was in their photos too. Then they saw who she was with, and were able to locate a friend of hers. From locating a friend of hers from this second photo, they located her.
Perhaps she was doing a presentation, so was alone at the front of the room. The photographer, being in the audience, would obviously know which presentation it was and from there its a quick look-up on the list of presenters.
yah I was actually thinking of facebook ad just after I replied, with that criteria..only thing that turned me off from it what that you made it sound as highschool was a post-processor (validation) vs pre-processor (targeting)
Taking the third definition of off merriam-webster online doesn't really prove your point here. There is a reason why the title of the op was "Why Engineers Scare Me". I know this was all in good fun but please do understand why this is stalking and how the same technique can be used in very harmful ways.
sophistry [ˈsɒfɪstrɪ]
n pl -ries
1. (Philosophy)
a. a method of argument that is seemingly plausible though actually invalid and misleading
b. the art of using such arguments
2. subtle but unsound or fallacious reasoning
3. an instance of this; sophism
Your reply appears to accuse the parent of sophistry, but you provide an even worse argument for your claim than the parent does. It is not obvious to me how posting the definition of “stalk”, according to which the bet was not stalking, is an invalid or misleading argument.
I am directly accusing the parent of sophistry by selecting to post a definition of "stalking" that required malicious intent if not violence when that is not the totality of the definition, is not consistent with Merriam-Webster or Oxford dictionary and seeks to excuse their behavior by the omission. .
The second photo you realized you had showed the subject holding the device used to take the picture sent to your colleague. You investigated that particular device and found a way to uniquely identify it's pictures based off some characteristic of it's photos, not necessarily EXIF, then scoured the Internet for photos with the same unique characteristic until you found one that revealed more information about the subject?
The team(including Justin: using first name here for convinience's sake) had a trip to Austin. Where they were should be a public knowledge. Perhaps they(you) started the search from there. A facebook search or so could tell some plans of people who would have fun at that particular place.
It is also possible that during the few days' correspondence one of the engineer guys had some other piece of info that we are not familiar with. Seeing how the engineers already knew her high school, I'm sure Justin talked about her with co-workers.
I think the key information that's missing here is what the engineers knew but Justin didn't know that they knew.(Sounds like a quote out of a Friends episode, mmm)
So you know the school and you have her picture. By her picture you can make a guess about her age. Then you search the web for graduates for particular dates from this particular school. It doesn’t have to be a yearbook or a class roster, people on LinkedIn mention the schools they’ve attended. Sooner or later you’ll find someone and then you can narrow the search. Even simpler, once you find someone you simply contact him with a fictional story as to why you’re searching this woman. From there on it just takes some old good fashioned social engineering.
You obviously knew something about her that made it possible, which you even admit Justin doesn't know you had. People are getting bored with your shenanigans, spill it!
I don't feel bad about letting the discussion continue as-is because of the fact that we completely expected to get the answer with only the information mentioned here. I still haven't heard this method mentioned yet directly.
The "extra" information we got came later after we already started working on it, and came as surprise. It just happened to bring the result back sooner than the other approach.
Nah I like this approach...let us bumble around for awhile in the discussion...I'd be very surprised if no one on HN could surmise the solution, but I'll admit I'm stumped (though I've only thought about it for a few minutes before getting back to work)
Can you or did you mention what it was that Justin didn't know you knew about her. Seems rather unfair, to him and us.
So we could all plow through every event that happened in Austin that she may have attended, but without knowing what part of town it starts looking rather boring. I have a real job. :)
So it seems you weren't even confident when you got your confirmation, a.k.a., it was an informed guess but could have just as easily been a girl that looked alike at an event at the time she was supposedly at an event?
So you stumbled on a piece of information which proved useful, notwithstanding that you didn't intend to look for.
If Justin observed correctly, you guys were very impressed with yourselves. Had you retrieved the answer by methods of which you were confident, you would've still been impressed, but not greatly.(I'm making a lapse conjecture here, of course)
Since you were confident that you would be able to find her, prior to looking at the picture, according to the article, I don't think you were looking for any information in the picture other than for confirmation.
However, some extra information in the picture(since it was taken very recently) enabled you to look further, which returned a result. Perhaps a building, an address, or some sort of landmark-ish location which narrowed whereabouts of the picture vastly? If this were the case, then I can understand that you guys were very impressed, because well, it is.
Wherever you were that night had a Facebook page with an event created for the evening you were there. You uploaded your second picture to the Facebook event, or tagged the Facebook page of the bar you were at. Then someone tagged the girl in the photo, revealing her identity.
Justin's own phone leaked his location (through some google-lattitude, or Yelp like service or perhaps gps directly) while Justin drove (or rode with her in a taxi) to her home.
How would they have gotten my GPS location? And how would they have figured out which one was her house. I was in her neighborhood and stopped multiple places that were 'significant' to her as a kid. Any of them could have been her house. And I wasn't at anyone for more than 60 seconds
At this point he could not see my screen (he was standing directly in
front of me), but offered me a simple bet – He said if I sent him the
picture of the girl he could find out her name.
Reminds me of:
Sky Masterson: One of these days in your travels, a guy is going to show you a
brand-new deck of cards on which the seal is not yet broken. Then this guy
is going to offer to bet you that he can make the jack of spades jump out of
this brand-new deck of cards and squirt cider in your ear. But, son, do not
accept this bet, because as sure as you stand there, you're going to wind
up with an ear full of cider.
> Simplest explanation: Someone saw you and told him.
Of course one of the first things I did was ask another friend who I thought met her. But no, ultimately the answer did come from the internet, not from anyone I knew.
- This was just a fun bet between friends. Justin really didn't think it could be done and I wanted to show him how a little information goes a long way online.
- Justin treated this girl well and she even knew about this bet.
- "Look up online" != Stalking
- None of the discovered information has been posted publicly online. This isn't even her photo.
> - None of her information has been posted online.
You are totally lying. I just saw your other comment on this where you admitted you posted her pic in a Facebook ad. I will quote:
>This is the closest I've heard to one of the ideas that we expected to work, so I'll share.
>- Create an ad on Facebook with her picture.
>- Target people that went to her high school within 4 years of the range of years we thought she graduated within. Was only a few hundred people so the cost of running the ad would likely only be a few bucks.
>- The ad copy / landing page just needed to be convincing that we weren't stalking her. We went with the "help us win a bet" approach but the "lost camera" approach would have been good also.
>Ultimately we got the answer sooner from another approach after we found out that we had another picture of the girl that Justin didn't know we had. So we canceled the ad, but I think this would have worked if given enough time.
I think the engineer was not stalking her in any sense that matters. Looking up information on someone could be considered “stalking”, but if there was no likely bad outcome from it, then it’s harmless. In the same way, sneaking into a movie theater without a ticket is usually a bad thing, but if you actually bought a ticket and then you accidentally dropped your ticket in the gutter, your sneaking in is moral and acceptable. It’s all about the expected result, not the means you use.
Why is “stalking” a woman bad in the first place? Because the stalker might get so obsessed that they get violent with or rape the woman. Or because the woman might feel like her privacy has been violated. Or the woman might feel nervous around the stalker, not knowing what they want.
None of these bad situations were going to result from this type of information-gathering. The engineer was not planning to hurt this woman or interfere with her in any way. The woman would not feel that her privacy has been violated, because they just want her name, and the woman probably wouldn’t mind if the bet-taker just told the engineer her name. And the woman knows what this “stalker” wants – to win a one-time bet. So she won’t be nervous about his intentions – winning a bet to find out a name is a harmless action that does not inspire worry.
As for posting her information online, yes, it seems like the engineer had an unintentional privacy leak in that they showed Facebook users from her high school her picture. (I don’t think he was purposefully lying about posting her info, I think he just didn’t consider the photo significant.) Showing the photo could be bad if the woman didn’t want anyone seeing that dress she bought, or didn’t want her alumni to be reminded of her. You could argue that showing the photo to those users was a mistake on the engineer’s part.
But you should keep in mind that the chances were pretty low that this woman’s picture leaked anything significant to her former high school classmates. It’s very likely that they didn’t care what dress she is wearing. Many of them saw her face already when she was actually in high school, and as for the ones who had not, seeing a normal photo of a random woman in a dress is unlikely to cause anything bad. There was a possibility, but it was very low.
You just contradicted your claim that no privacy violations would occur.
You say no privacy violations "were going to result from this type of information-gathering". However you then concede that it resulted in a "privacy leak in that they showed Facebook users from her high school her picture," mentioning several reasons why the target may find this invasive. You frame it as "unintentional", but that is irrelevant; you've contradicted your claim.
I would certainly not argue that the critereon is intent to do physical harm, which is how you define "stalking". Boy, would an awful lot of stalking no longer be considered "stalking" under that definition. By your definition cyberstalking is ok if you only want to date them and/or joke with your bros about it, not rape someone.
Rather the issue is whether the actions of, say, posting your photo in a Facebook ad campaign -- and otherwise pursuing means of "information-gathering" so extraordinary that one needs to post to Hacker News to explain them -- might reasonably considered a willful invasion of a person's privacy. Certainly it could be, and these guys just didn't care, because they're just having a laugh, and she's probably cool with it, and if she's not, well screw her because girls should be cool with that sort of thing.
It's interesting you claim the word "honest". You lied about posting her picture online. Not only did you post it online, you targeted her classmates and let them know she was flirting with some douche in your company!
Yeah, no one would ever possibly be creeped out by that.
You think it's ok because you have good intentions. Guess what? So does every other creep. It's not up to you to decide whether your actions are frightening. You took a bet that you could stalk (message all her classmates!) and dox (post her picture online!) a girl without her permission and without even the slightest self-awareness that it's invasive behavior. And then you tried to lie about key details.
You, sir, are exactly the kind of male-privileged "brogrammer" that is giving this industry a bad name these days.
I haven't lied. We were talking about different things. I was referring to the fact that throughout these discussions none of her actual info has come out and that we've been careful not to post any of the information we discovered about her from our search. I edited my comment to try to make that more clear.
Also the ad didn't say that she was flirting with anybody and it was designed with an attempt to not make her look bad in any way.
Finally, she is aware of the whole deal and has been a good sport, and is not upset.
You guys absolutely have written about her flirtations in this blog post, which everyone who saw that Facebook ad can now link back to her.
You said you didn't post any of her info online.. you ran a friggin Facebook ad campaign with her picture on it (which you obtained not from her)! Now you've just edited your comment to say you didn't post any other info about her that you obtained.
You did all this without her permission, according to your colleague. Now you're trying to weasel out of that too with ambiguous phrases like "she is aware". Yeah, after you told her what you already did.
Did it not even occur to you that it could be potentially embarrassing or invasive? What if she wasn't cool with it? What if she's just playing cool because you forced her hand? It's wrong of you to assume it's ok to cyberstalk someone for "fun" because your intentions are good -- your intentions being having a laugh with your bros.
The other point here, even if you think bro'ing out like this is perfectly acceptable workplace behavior / way to treat women, is that it's just incredibly stupid for a startup to tarnish their brand this way. Just don't do it.
> - This was just a fun bet between friends… she even knew about this bet.
No, from what you guys have said you didn't tell her about it until after the fact. That's super douchey and retroactive permission doesn't change anything. If you grabbed a girl's butt and she laughed it off as "fun between friends," it doesn't make it ok. You need to ask permission first.
If you think cyberstalking is only bad if you add doxing to it, then you don't understand cyberstalking.
Calm down. The engineers were proving to the sales guy that it's possible to get this kind of information online. They weren't seeking any advantage over the subject at all, directly or indirectly.
Keep in mind that the information that they were looking for could just as easily have been volunteered by the sales guy in other circumstances, so it's not like this information was tightly under her control in the first place. "What's her name? Where does she live?" - these are pretty common questions when discussing romantic liaisons with friends and colleagues.
She knew what was going on, not just after the fact.
I've just re-read the definitions for "stalking", "cyberstalking", and "harassment" and I can assure you none of what we did qualifies given the actual scenario of how things really occurred.
Did you tell her about it/get her permission beforehand? Or did she say something when you met her that indicated it would be okay? If so, great. If not, this is still in the creepy / douche canoe area and the fact that she happened to be okay with it after the fact doesn't really change that.
Yeah, this was my thought too. This whole situation is just creepy and its even creepier that so many people have a million other ways they would stalk somebody. Sure you can do all of these things, but if you have ethics you just don't.
I feel like the majority of folk are treating this as a purely intellectual challenge. Which it sort of is (to the HN crowd, since the actual photo information is missing) but please, let's not forget the context here.
So you're saying that http://close.io not only has good engineers but also is good at marketing? Maybe you should check it out if you're looking for sales software then :)
1) He did not say either group was 'good', and implied quite the opposite.
2) As someone who purchases things like enterprise sales software, this exercise does not entice me to consider your product. Ever.
I was thinking the same, and expected the answer to be something like "Phil used the API provided by Close.io that invert google picture search and was about to get her name", but it turns out Close.io doesn't do anything related.
But yeah, the post clearly emphasizes that they have smart engineers.
I'm with you. I bet these guys though 'titstare' was fracking hysterical, complete with fistbumps. They certainly have the whole "whaaaaaat...we didn't do anything wrong...allow us to rationalize this away" thing down.
If you know the high school and year(s), you go thru the friends list of every person on Facebook who did post their high school from that year (or nearby years), and does show their friends, until you see her face.
That's only a couple hundred kids, each with a couple hundred friends.
If you still get nothing, start doing friend requests for the people that don't show their friends. I'll bet you'd get another 25-30 kids that way.
I don't see why you'd need facial recognition software to go thru 20,000 pictures.
3/4ths of them would be the wrong gender/age and be immediately disqualified.
But if you're a programmer, you can probably scrape and aggregate to speed things up - drop out the males, other high schools, only show pictures once, etc.
I think what makes this feel creepy instead of cool is that you've just described an exploit without giving any thought to mitigating the threat.
So you've found an attack vector where you can get a woman's name and address off the 'net with a picture and a small amount of information. Maybe it doesn't work all the time, but it worked at least once. What are you going to do with that attack vector? Are you going to make an iPhone app, so that others can snap pictures of random women and recreate your exploit automatically? Or are you going to come up with ways the attack vector can be shut down, things that either individuals (eg, the woman herself) or organizations (like Facebook or Google) can do to block the attack?
There is some redeeming social value in publicly defusing paranoia.
Coincidentally this is 9/11 day... It turns out that anyone of room temp or above IQ can easily do something fairly awful, but the fact it almost never happens provides some faith in humanity that basically no one (on a statistical basis) is actually awful.
> What are you going to do with that attack vector?
Huh? Nothing... this was a just a fun bet between two friends with no bad intentions. I didn't discover anything, as evidenced by the dozens of others in this discussion with similar ideas.
One of the things that looks like a clue to me is that this the lead engineer of close.io. Browsing through their site, it looks like they do "telecom in an API"; it reminds me of Twilio and OneBox.
So, here's a thought. Let's say the Engineer has access to the SMS logs (NSA-style) for Justin's phone. (Maybe his phone goes through their system. But even if not, maybe it's company-provided, so there's an admin interface provided by the telecom that lists this information.)
Justin probably doesn't regularly text people in Texas, since the team is based out of Palo Alto. So look through his logs for a number in a Texas area code (http://en.wikipedia.org/wiki/List_of_Texas_area_codes) that he's been texting more often than usual.
That gets you the girl's phone number; probably a cell phone. Now, you need to go from the cell phone number to a last name. Reverse Phone Lookup probably won't work for this, although it's certainly worth a first try. More likely candidates: looking through the users table at close.io for a matching phone number, or doing a google search for the phone number. Or, try calling in the middle of the night and seeing what the voicemail says. Any of these approaches might work.
Once you've got a last name, grab a copy of the white pages. Most public schools publish their district boundaries, so go grab a copy of that. Look through all entries that match the last name, and see how many of them fall within (or close to, in the case politics made the boundary change) the district boundary. You probably will get a handful (maybe three or four). If you only get one, bingo!
If you get more than one, I'd call the cell phone to see if I can get a first name somehow. Then, I'd go through the landlines in the White Pages and call them one at a time, "Hello, may I speak to X? Sorry, wrong number." until one matches.
I love this explanation. But unfortunately there was no telephony hacking of any kind as this was just an ordinary personal cell phone.
I did have an idea that we could try to hack his Verizon account to look at his SMS log, figuring that Forgot Password questions might be easy or that we could intercept an email. But we didn't do this.
No need to call in the middle of the night to see what voicemail says. Use slydial.com and it connects directly to voicemail without ringing. If you hang up promptly, they get no record of a voicemail being left. I do this for most unknown numbers which call me.
In additional to using TrueCaller.
Since the point of this article is probably to promote their company, having the answer be "our engineers looked at your private information for a prank" seems like a no go.
As someone who happens to be good at finding people on the internet (true confessions), the approach I would have taken would have been related to FB likes, or log-ins at the bar that they met on Facebook. I.e. they were all at the same bar in Austin, so search for people who have checked-in at the bar. And perhaps the other photo the engineers had was another location that she liked, or another piece of information about the bar in question. Then, it's just a matter of narrowing down results based on age and location. The fact that they knew the high school would have made this quest even easier. I once found an individual from a first name and the name of the bar they work at in (comically) downtown Austin. It took forever, but I had patience. :) The internet has all sorts of shortcuts to finding people if you know how to look!
If they knew her high school and were doing lots of batch processing, I wonder if they found digital copies of the yearbook and tried facial recognition. For a single subject, it doesn't even need to be particularly accurate- they can easily sift through twenty possible matches with their own eyes.
She looks pretty; At least for guys pretty = memorable. When I was single I was pretty bold with the ladies and I'd have flirted with her first (perhaps getting shot down, oh well), remembered at least some demographic information, and then used it to completely F with my coworker's mind for awhile. When I was younger I was quite the practical joker.
Note the convoluted written responses about not talking to anyone he knew and so forth which technically does not exclude the girl, herself, saying her own name.
The other alternative is she's somebody's buddy/family member so it would be trivial to know before hand that she's so and so's ex or sister or whatever and given that intel, figure it out.
I assume the author is not a Facebook friend with the girl. Otherwise it would have been too easy.
But friends of friends is not the only privacy leak on Facebook. There is a very old feature (which is about to get deprecated) called Networks.
Networks are created for schools, universities and other big organizations. They had the name of the high school so they have to list it as their own high school in their Facebook profiles.
This will automatically put them in the same network as the girl. I am assuming here that she has listed that on her Facebook profile.
Then performing a graph search for women who has gone to that high school and live in Austin with an age between X and Y will narrow the search a lot. Even if some of that information is not public on Facebook it would be public to them if they are not same network if she has the default settings. This is a neat trick which is still usable.
I don't think looking through the pictures would take more than half an hour.
Is it just me or is this spectacularly creepy? "I bet you can't cyber stalk my friend! I bet you we can!" I could understand if this was at defcon or a security/privacy related conference and all parties involved were in the field, but just cyber stalking some random woman? What's the point?
No, it's not just you. It's spectacularly creepy, lacking reflection _and_ a thinly disguised ad for their site/product ("look, our engineers are fucking evil geniuses! Amazing."). And of course, like others already have said, HN fell for it.
This stuff should worry you, not be lauded as some kind of amazing engineering feat. 4chan does this all the time and they call it doxxing.
Assuming that she didn't show up in a reverse Google Image Search, maybe they used some sort of facial recognition API (like http://www.lambdal.com/), perhaps along with the other people in the picture, against scraped pictures from LinkedIn and Facebook based on her appx age, location (Austin), gender (F), and high school.
I think the key is knowing what high school she went to. From there, you could possibly get yearbook pictures and student lists. Knowing an approximate age would narrow the possibilities significantly as well.
I would probably take a social engineering approach to this problem- if you knew her high school and approximate age you could probably find someone who was in that class who would know her pretty easily. At that point it's just a matter of either tricking them into identifying the photo or just straight-up asking them.
So basically the engineers knew something specific about the girl already which narrowed their data set. Unless they tell you what that was, we can go all day long guessing how they "figured" it out. In fact with this information I am interested in the actual time taken to find the girl. They could have found her in five minutes after receiving the picture and you wouldn't know. Why? From the article:
1) They were running software on a computer for hours
They are engineers!
2) The first couple of tries didn’t work
Are you sure they weren't debugging?
3) They knew what high school she went too prior to searching
If they know the high school they only have to find one or two established teachers who were around for a long period of time. It is not unlikely they can call a name from a picture after all those years.
This smells like viral marketing to me (too bad I still don't know what close.io is, should have added some information to the article. Or are you looking for new engineers? "Hey look! At close.io we seal bets with engineers. About girls! Come work for us!").
Possibilities:
- they looked at signups at close.io after the evening, thus narrowing a list of female names for image searching. They could further limit this to IP's from the austin area.
- They bought the ad words 'justin gold', and 'justinbgold'. Then waited for her to google you, thus revealing IP addresses for further googling.
Saying she "doesn't exist on the internet" is a big assumption. First, she is on Facebook, so she's on the internet, easily accessible or no. Second, most people leave some internet trail through online forums, newspaper articles and newsletters.
I find it very likely that she was found on the internet somewhere.
I'm guessing it has to be something to do with facial recognition, since he asked for the picture without knowing its contents (other than that it has her face in it).
From there, if he had an idea of where they went that night, a club or bar, perhaps he could do some Facebook/Twitter searching magic to search for pictures of that night, taken by other people. Then they could look for her face in the background.
Not sure where to go from there... perhaps they got lucky with one of the pictures being tagged with name. Or perhaps they identified everyone she was shown to be associating with, and found intersections in their Facebook friends list? Does it have to do with Facebook?
How are you surprised that they did it when they knew what high school she went to? Two possibilities:
Option 1: Step 1) Find copies of yearbooks going back a few years, 2) find her face in the yearbook.
Option 2: Step 1) Search facebook for people who went to the same high school as she did in the probable time frame. 2) Look for her in that set, or in the set of friends that the people who matched the search have.
Feel free to sprinkle in face detection on the yearbook/facebook photos to make it more engineer-ey.
As a matter of fact, how did they find out what high school she went to? I can't imagine you telling them that if you were not providing much information about her to anyone else.
They knew what high school because it was a slip up. If you knew these guys, they aren't the kind who would just sit there and look at pictures. They had some software running.
> If you knew these guys, they aren't the kind who would just sit there and look at pictures.
As someone who has been a software developer for quite a long time, this statement makes a big assumption.
A good software developer knows it is often far easier and faster to flip through a couple of hundred photos and process them with your meat-brain rather than write a one-off bit of software to do the matching, even if you start from an existing base like OpenCV or whatever.
Having software running doesn't preclude using your brain - first to consider possible heuristics and then to do the types of analysis computers aren't good at. There's a reason mechanical turk exists.
jbg331 - while we wait for people to ferret out the right answer, can you tell us how the woman in question has responded to the "game"? Genuinely curious.
Although this most probably isn't the case here, note how making a post impersonating the person who'd lose the bet would be a great way to solicit methods to win the bet.
This thread kind of exists as it does based on the fact that people were assuming there was something clever and software related that lead to the information because that's what the sales guy thinks, but the incomplete information we have from the developers suggests that they simply had a second photo that probably had some kind of identifiable information in it like her name that they saw via ZOOM,ENHANCE and a bit of Google which anyone could have done, even the sales guy.
FWIW, as others have pointed out, I do find this a bit creepy too, on two levels. First, the initial bet is a bit creepy. Guy involved said girl laughed it off, but that doesn't necessarily mean she didn't find it creepy that some guy she recently met was betting other people they couldn't track her down Enemy of the State style. And then on top of that, inviting the entire Internet into the game via the second obscured photo adds a much bigger layer of creep on to it, especially since in the process a lot of secondary information on her was leaked. Probably enough along with the photo of her face missing for others to find her, and you've kind of made that a public challenge indirectly. (Not one I have any interest in pursuing because like I said, the whole thing seems a bit creepy to me).
Are you sure? (serious question here, since I'm not) philfreo said:
"This isn't the actual photo but looks similar, except that she was alone. Justin was afraid of reposting the same photo after the results of handing over a photo last time :)"
Which to me suggests it is the same girl in a different photo. I would still find it creepy even if the photo is not of her at all, though somewhat less so.
The photo used in the story is not her. I am 100% certain. The photo in the blog is a picture of a random girl who lives in another part of the country, has different physical attributs, and is a different age than the original girl in question
Well everyone's on his/her phone these days, so let's assume a second picture caught her busy with her phone. Highly doubtful a photo would be high-res enough to get text-level detail but certainly the general layout and colors of whatever she was looking at would show up. So if this led to a successful ID, she must have been looking at an app or page that is user-customizable to some extent like facebook with a unique profile and cover. If you have even a blurry image of a website, the RGB colors are still going to be pretty close so if you have some sort of way of automatically iterating through user pages on the website in question (GraphAPI), doing direct page capture to image, scaling down that image, doing the appropriate rotation, shear, etc., and then running some simple image match criteria, that "might" work. Facebook has billions of users though which surely would take more than 24 hours to iterate through, so maybe you could prune the search space by only searching friends of people claiming the high school graduation year in question.
- using tools like http://www.pictriev.com/ or similar offline solutions to get a pretty good estimate of her age.
- using image analysis to "fingerprint" the camera used to take the picture. Then crawling sites where she would be likely to post pictures (twitter, etc.) based on the suspected area and comparing to find matches (i.e. pictures taken with the same camera). I'm not sure what the current state-of-the-art is for those algorithms, but if for instance the camera displayed an obvious visible flaw (darker spot), it would have made it easier. On a perfectly clean camera without visible defects I assume that more than one picture is necessary for effective camera fingerprinting.
If your engineers as self-satisfied about finding her as it sounds, then they'll be happy to describe their process over a beer or two. I've never known an engineer who doesn't like discussing how they solved some riddle.
The engineers were made to beg for info on the girl, so they might be getting some satisfaction from keeping a secret from the author in return. I know I would. Give 'em a taste of his own medicine. Also, if it was a relatively simple solution, it's probably more exciting with some mystery overlaying it rather than just explaining it outright.
Where was the photo taken? What was in the photo other than the girl?
If the photo was taken at her home and there is any landscape in the background, the obvious initial point of attack would be to identify the location the photograph was taken.
Made an ad campaign on Facebook targeting her high school and girls of her age saying in the ad something like "Click on this link darling, need to see this, Justin Gold ;)" and redirecting her to some fake (made by you) rosy website with hearts on them where she is socially engineered to leave her name.
Probably not this, but I would try it for the fun.
Philfreo > I think that the girl did share something with you that Justin doesn't know, such as something that she did accomplish in that school, like winning some kind of award, winning a competition or something similar. Then, you used that information along with the name of the school in Google.
1. Compose tactful email text explaining the bet. This is optional if you don't care about tact or a potential negative impact on your or your company image.
2. Go to fiverr and get a person or three to find teachers, administrators, and students who were at the same high school around the same time. Have them send the letter you composed to these potential teachers, staff, and peers. Track with close.io.
3. Get answers and try to confirm results.
4. With new data, iterate text of email and/or task request for fiverr if appropriate.
If you are skilled at chatting up gatekeepers (many sales folks are good at this), then you can call the school admin, explain the situation to her, and the answer would be yours in a few minutes.
There are some other options that are of questionable legality and/or ethics, but I will stay away from those.
I wonder what the resolution of the original image was?
The inside of her thumb is visible through the glass, so with enough detail it might be possible to record her thumb print. Even if you could do that, though, odds are against her print being present in any publicly-available database.
This is not a marketing ploy and has nothing to do what our company does. This was a bet between friends, and everyone I told this story to got a kick out of it. That, plus the fact I hoped someone on HN could help me solve this puzzle led us to post the story.
The answer is probably simple. Social engineering. Find someone who went to the high school, then send them the picture and ask who it is. Concoct some reasonable non-creepy story like "I found this person's wallet and I want to return it".
Knowing the high school and approx age in Austin...
You probably know the year she graduated, you can run subsequent queries using +-1 based off of that graduation year. What you're after at first are rosters. The end goal is to place a picture for each name discovered.
Over 10 years since high school graduation? I'd be looking at one of the classmate or facebook groups regarding reunions. Less than 10 years since graduation? I'd still look trying to compile a list of names from these places. Old high school sports and club rosters maybe? Build a graph of people who did go to that high school during those years. Start building 1 degree of separation.
Once you've gotten a list of names, start cross referencing local groups, meet ups, colleges starting with the University of Texas -- possibly even spamming people using generated you've collected )lastname_firstletter@utexas@edu). Perhaps a phishing Email as follows "information regarding blah blah, looking for the person in this pic, we think they left their purse/cell phone etc"? Or post something in a local Austin Reddit.
I'd also do simple searches based on individual names and generated college emails. Start cross referencing all college clubs past/present with names and/or generated emails. If the person is/was in graduate school they will probably have even more school related information online.
If the person has graduated, start working the job route. Online career profiles monster, linkedIn, facebook. Have they done any sort of volunteering? Lot's of pics associated with volunteer work are out there.
5K results are a treasure trove, location, name and age ripe for searching. Some 5K's have corresponding flickr sets where you can match name to race number to a face. Once you have faces either look at them or run them through some CV for possible matches. Even if the name you've found isn't the person you're looking for such pictures might lead you to them anyway, if that person appears in a picture with your target. If you know that the person you found knows your target, start looking at that person's information. Who do they follow on pinterest, twitter, etc...
Does this person own a house do searches against the tax databases for full address of home. Even if the address isn't useful you might learn their middle name - re search with middle initial and middle names in lieu of first names.
Look at old social networking sites like myspace - the target might have old profiles up?
Did you recently add her on any social network? Even if her FB is inaccessible, yours isn't... maybe a friend recommendation engine would eventually suggest her if someone just friended and unfriended you repeatedly?
Did anyone mention twitter ? Maybe seing her tweet at some time and looking for tweets happening at the same time in some region lead you to her tweet account. Then from a tweet account to a blog, then to her name ?
In the actual photo? Is there any meta data? Is there any EXIF data? What was the photo filename, and what does searching for that (with increments up and down) return?
To try and guess what happened:-
1) One of them followed you and got information. (You say this didn't happen.)
2) They had information about the event and worked from there.
3) There were other people in the photo, and they got face-matched, and their friends lists were not hidden, and they got it from that.
4) They had snippets of other information and just trawled through very many searches to get the name.
I'm a bit confused how they didn't already have her name. You told them you met someone, but didn't give any kind of name?
Hate to sound cynical, but this seems like an attempt to solicit responses to identify good engineers to hire or promote some upcoming feature of the web app.
Did you use Tineye or Google Image search to see if there were other photos from the event? A combination of her dress and the time of her arriving there?
He said she only bought the dress recently. They did a image search and found a local store that stocks that particular dress - and she had checked in with foursquare?
Actually, if you read the fully reply, that's NOT how they did it. They ended up finding her via another approach. Phil mentioned the other approach which he thought would have worked.
I'm sure there's a lot of information missing from this story, which leads to a lot of possible solutions that aren't negated by the lack of limiting circumstances.
More than likely the author is an idiot and the image had EXIF data. Considering they already knew her highschool, it wouldn't had been to hard to go through the year books in a few hours.
Heh, geek blog effect. Reddit et al have created some very bad habits of dialogue, such as assuming the subject of a link isn't reading it. And being nasty with candor.
I still remember the time I said something impolite about David Brin. Brin took offense and replied.
I'm actually a huge fan of David Brin. That sucked.
@asymetric - I am not an engineer, I don't know how a lot of these things work. Rather than critique, I would appreciate a better analogy that I can use in the future
HN tries to avoid that kind of rudeness, but it happens. Like I said, geeks can get bad habits from the locker-room culture online. You'll notice that your critic's posts are gradually fading to grey thanks to all the downvotes he's accumulating (established users get a downvote button).
I'm not sure where you go the idea that pseudo-celebs and IT managers are immune to stupidity or posting stupid articles. There's always an element of "who cares if my content sucks, lets just get it done" when posting bloggy stuff, and it's pretty apparent here.
Calling EXIF "back end stuff" is pretty stupid, I don't care if the author's ego is bigger than that statement, it's still true. The article might as well be how many apples fit in a bag or how long is a string.
The same two engineers who found out who she was? :-)
Seriously: given your story, if, say, they are colleagues, too, they may be in on it, and lie to you about the EXIF data (if they _are_ engineers, they certainly lied; how can there ever be 'nothing interesting' in EXIF data :-)?)
Most of the time, the simple answer is the likely one. My money is on the high school-yearbook line, but it might have been anything (did one of your colleagues go to the same school?).
A photo without EXIF data? Someone must be trying to hide something from me. Let's see how good he is, and check whether there still are parts of the file with metadata in the disk's free blocks.
"or replaces it with metadata from another photo"
A Nikon F5 at f/5.6 and 1/100s? No way! If so, that car must have done a thousand kilometers an hour or so. Also, the aberration looks more like that of an Canon lens, but I'm not 100% sure of the model. I wonder whether it is possible to train a model on the 'JPG to camera model' problem.
"or makes up some bogus metadata."
Hm, I thought the Eiffel Tower clone in Japan was in Shanghai. The GPS coordinates seem to indicate that Tokio has one, too. Let's google to check that.
OK, that's more for the hacker engineer, but I thought one would not have to make that explicit on HN.
>Someone must be trying to hide something from me.
Umm, No, many popular photo-sharing sites at least give you the option to strip metadata from photos when posting. IIRC, fb does this as well, by default.
> and check whether there still are parts of the file with metadata in the disk's free blocks.
Good luck.
>A Nikon F5 at f/5.6 and 1/100s? No way!
It doesn't have to be believable metadata to be practically useless.
>Also, the aberration looks more like that of an Canon lens, but I'm not 100% sure of the model.
> I wonder whether it is possible to train a model on the 'JPG to camera model' problem.
Sure, but not for beer money. You could probably even have some success identifying individual cameras by characterizing their CCD/image sensor, and comparing to known photos.
The fact that you required two engineers to do something that you could have confirmed yourself with 2 seconds of Googling shows that your ego is a bit inflated.
This is dumb. You can get a private investigator to do this for $100 - $200, with almost surely a better method that Phil used.
Given what Phil does, the most cost effective (rational) method he could have used was to spend those hours working, and then pay a private investigator.
If he wanted to do it himself, the easiest way is to facebook search for people who went to her high school in the right date range, then ask them who is in the photo. Given the start time and end time, Phil probably didn't do that, he probably browsed their photos & friends until he found her.
Not a very good use of time, but it would work.
I think guessing the exact method used isn't interesting, because the problem isn't a particularly hard one. He's done the social engineering equivalent of picking the lock on your filing cabinet.
- This isn't the actual photo but looks similar, except that she was alone. Justin was afraid of reposting the same photo after the results of handing over a photo last time :)
- There was no useful exif data in the image.
- We knew her high school and approximate age in Austin, but were unable to find any yearbooks / class rosters online. (Idea not pursued: try to acquire a yearbook offline).
- Facebook Graph Search and LinkedIn with the criteria we knew didn't help.
- Google Image search didn't help.
> "I'm sure there's a lot of information missing from this story"
Quite true.
Ultimately we had several methods going at once to try to figure it out.
The one that came back with results first relies on some information Justin still doesn't know we had (though he would still be impressed), but other approaches we also expected to work would be possible with only what's in the story, and haven't yet been mentioned here.
EDIT: shared more at https://news.ycombinator.com/item?id=6369751