Who's watching the store?

[scruge]

If you load those areas into digle and then drop the Qos 1 notch, you'll find they all go away, except a small hand full in the urban areas and a dozen or less on the rural road.

True, but unless we go back and resniff the same area a second time, most all the stuff that is found brand new will have a QoS level of 1 (assuming 1 is lowest). That's why I'm hoping Israel will eventually sniff through the areas that I ran through in Los Angeles, to raise the QoS level of them to something other than 1.

I see your point, but I have trouble believing that no one before was able to detect all those APs in a small town, especially those that are so evening spaced on the highways.

[israel]

I see your point, but I have trouble believing that no one before was able to detect all those APs in a small town, especially those that are so evening spaced on the highways.

I am still interested to know whether or not anyone else has been down there to verify or discredit these issues? This would certainly be the fastest way to know for fact. Perhaps there is a system running out there that requires even spacing of APs for some driverless car systems. I sure as heck know that if something like this was somewhere within a hundred miles I would give it a looksee.

Israel Torres

[uhtu]

occam's lazer suggests that its bad data.

the question is, and remains (and what we're trying to find out) is: bad parse? bad wigle-side data handling? bad client? legitimate client with bad enviornmental impact (stumbler being followed by black helicopters with rapidly changing MAC on their AP, etc.)

that something is improbable dosen't make it impossible. that something was only seen once dosen't mean it wasn't seen. its just not always a nice and reproduceable observation like Good Science^sm would like.

we're continuing to look into this, and would still appreciate any constructive observation/insight :-)

[ccie4526]

that something is improbable dosen't make it impossible. that something was only seen once dosen't mean it wasn't seen. its just not always a nice and reproduceable observation like Good Science^sm would like.

As I noted in my email to wigle-admin, it really looks like someone went driving down the road with a second laptop running fakeap, spitting out a new mac every 30sec or 1min or something like that. I experimented with that a couple years ago, it's really quite easy to configure and make operate.

After my father and I had driven around downtown Houston, remarking that it was so easy to just sit on a street corner and pick up new APs,
I had heard a fairly reliable rumour that someone who works in downtown Houston had set up fakeap and left it sitting in his office window for several months. Thus when I started experimenting.

Knowing that it's not kosher to taint the database, I have specifically not used fakeap since, although several of my drives through high-density neighborhoods have led to subsequent accusations.

Thus, it is likely a legitmate client with bad enviro impact, just based on the mapping data.

Interesting question, do the admins have the capability of doing a "found only by me" search on any user in the system? It may be interesting to check a couple of the higher count usernames (I have my suspicisions, but will NOT air those in public discussion) and see if it can be pinned down to a user, and thus a specific file or files parsed.

My $1.00/50 worth.

[israel]

It may be interesting to check a couple of the higher count usernames (I have my suspicisions, but will NOT air those in public discussion) and see if it can be pinned down to a user, and thus a specific file or files parsed.

I have always wondered why submitted data isn't digitally signed so that it may not be tampered or generated programmatically which all the logging currently allows for. Sure people would be able to tinker with obscure patching to bypass this type of security, but most users would leave them by default. The idea is to keeping the honest uploader honest and sway away from making it so easy for anyone to taint the database.

Israel Torres

[ccie4526]

I have always wondered why submitted data isn't digitally signed so that it may not be tampered or generated programmatically which all the logging currently allows for. Sure people would be able to tinker with obscure patching to bypass this type of security, but most users would leave them by default. The idea is to keeping the honest uploader honest and sway away from making it so easy for anyone to taint the database.

Scenario: I drive around with fakeap and create a few thousand entries. I digitally sign those files, and submit to wigle. Boom, tainted database.

I think part of this is going to have to be a function of honesty of the users submitting the information, and self-policing of the users, and the users calling bullshit when we think tainted information has been submitted. If we can determine that a group of tainted information has been submitted, do the admins have the ability to confirm this tainting, and the ability to remove that dataset from the database.

Once again, my $1.00/50.

[israel]

Scenario: I drive around with fakeap and create a few thousand entries. I digitally sign those files, and submit to wigle. Boom, tainted database.

Yup, I was talking with someone the other day regarding this type of issues and we came to the conclusion if someone really wanted to they could disseminate really authentic looking data that was totally bogus, and it is likely that no one would really know ever. Sure everyone can weed out the obvious taint-attempts, but anything short of being obvious automatically gets trusted. As the saying goes: "Where there is a will, there is a way." It would probably be more significant if there was monetary gain from submitting APs (which there isn't) which is usually the motivator of inspecting every log submitted.

Just thinking here... isn't riding in the trunk with FAKEAP obvious when looking for the same SNR values being pumped into the logs since the distance of the FAKEAP and the sniffer never physically change? Maybe building an array of these units would camoflauge this detection with a hallway illusion.

I think part of this is going to have to be a function of honesty of the users submitting the information, and self-policing of the users, and the users calling bullshit when we think tainted information has been submitted.

Yeah this is very similar to the eBay model. If you can't trust the people (bidders/sellers) then the site becomes untrusted and therefore cannot function as intended.

Using QoS as a judgement tool is a very slow process as the world isn't populated with a high wardriving ratio. I think it would be a cool idea to have volunteer groups that the WiGLE team can call upon to verify areas when there is a dataset in question. Currently location does not appear to be disclosed as part of the stats (and maybe it should not be), but I am sure the admin know the homes of many of the submitters and would just need to check the patterns, but asking is always easier. Using the location information and the volunteer (investigator) status they can ask a person to wardrive a particular section or even specifc address to see what is up. I doubt the WiGLE guys have the time or resources to single-handedly travel to an area and check it out. Of course I volunteer myself within a 100 mile radius in case the need ever arises.

Israel Torres

[uhtu]

well, this isn't a surveillance site. we don't know anything.
the complexity implications of a fully trusted anything are.. you know..
complex.

what wigle does is aggregate observations. that's it. if those observations are wrong, the aggregate is wrong. q.e.d.
if a thousand people submit that a given identifier was seen at (x,y) that's all that means. it says nothing about where that thing really is.

confirmations, net-hugging strike forces, etc. all seem a bit overkill-ish.
myself, i don't really pay much attention to anything with a low QoS, but i don't use wigle for anything other than the bizzare maps that come out of crunching on the data.
(well, that and the wildly interesting people that intersect with us.)

i like the idea of a "bullshit" metric although i can't really think of a good way to go about doing it. positive reinforcement of data (combined with a form of filtering) is much easier than negative reinforcement. and we're so very very lazy.

perhaps a positive inclusion/weighting system e.g. "points from bobzilla are more 'trustworthy' than points from uhtu because i know uhtu had a broken-ass gps for 18 months" which would be interesting but complex, as it requires view modificaiton per-user rather than view exclusion per-user (like a broad QoS filter) that, and it'd just kill the current system performance-wise. and not really tell you that much more.

we're happy to evaluate any solutions that folks might have, but it really is a broad problem that we haven't seen any quick fixes for..

[ccie4526]

Hmmm, I would observe that an unnamed person no longer appears in the stats list, and that the webmaps for an unnamed area now seem to no longer show a bunch of APs that we all tended to call bullchit upon. Someone finally crossed themselves afoul of the wiglegh0ds and found themselves snuffed out in an instance?

Woot!

[electrojc]

I have driven some of those roads before and I know there was not that many ap's. I'm glad to see it got fixed!

[uhtu]

the dark lords of wigle^r^tm^sm have enough internal crossing and fouling for everyone in the known universe, all betwixt ourselves (*shakes fist at bobzilla*).

the wigle dataset is an interesting problem; a very very large aggregate of almost universally bad data. however, graph the numbers of any system and patterns emerge. some of our patterns are pretty maps. mmm.. cartographalicious...

we work fairly hard (for lazy people in their Copious Spare Time) on the continual evaluation of the data - both existing and incoming - to find new and better ways to use it to generate electricity which we can apply to advance Science^tm (usually by way of new and better PokingSticks^sm.)
one such new approach (which has been in the works for a while) was taken over the weekend, resulting in the re-evaluation of a modest (relative to the whole) number of points in the dataset. a number of users' postings over a fairly large swath of space and time were affected, due to the scope of the filters.

we'll try where possible to keep these kinds of things as ingress operations, so there's less juggling due to post-facto cleanup activities.

thanks again for your collective patience. you keep on stumbling, we'll keep on shouting at each other in the Wigle Dark Chamber of Evil and Mapmaking.

[arkasha]

I hate that place. I keep tripping over the corpses on the floor... dark and evil can be a perilous combination for decor. But the maps tie it all together.

[wrzwaldo]

Hmmm, I would observe that an unnamed person no longer appears in the stats list, and that the webmaps for an unnamed area now seem to no longer show a bunch of APs that we all tended to call bullchit upon. Someone finally crossed themselves afoul of the wiglegh0ds and found themselves snuffed out in an instance?

Woot!

You wouldn't be talking about #30 would you?

[pejacoby]

Aha, does that account for me seeing the stats at 4.9 million a day or two ago, and now at a mere 4.77 million?

Or maybe it was all that cough-syrup...hmm....<hick>....

[ccie4526]

You wouldn't be talking about #30 would you?

<cough cough>