WiGLE currently has 372 files queued for processing

Notes regarding project status and info

15 posts • Page 1 of 1
Someone might want to check on the queue.

WiGLE currently has 372 files queued for processing

as of 20060824 @ 8am, PDT

thx
-NB
WiGLE currently has 372 files queued for processing
I see items get removed from the queue, it's just verrrry slow so there is a buildup of entries.

The dual-hamster setup for keeping the site running must have one hamster sick or very tired. Please, think of the hamsters!
WiGLE currently has 372 files queued for processing
I see items get removed from the queue, it's just verrrry slow so there is a buildup of entries.

The dual-hamster setup for keeping the site running must have one hamster sick or very tired. Please, think of the hamsters!
No no no.. It's the interns.. Or rather the new intern. Her name is Monica and she is busy with the server admin. You'll see the stained dress on the front page when the investigation starts.

Dutch
[url=http://www.wigle.net/gps/gps/StatGroup/listusers?groupid=20041206-00006][img]http://home19.inet.tele.dk/dutch/netstumblerwigle.gif[/img][/url]

Postby whitedice » Thu Aug 24, 2006 5:01 pm

67 files parsed. 0 new networks today. After 12 hours?

Postby mahlerrd » Thu Aug 24, 2006 5:14 pm

I wonder, as a curmudgeon who doesn't like to attribute to coincidence anything that could be explained in another manner, if the intern's new "garbage-finding," ranking and triangulating algorithms might need a bit more testing?

Or, as is usually the case when I write something, is there something in the dataset that is confusing their algorithms?

Postby ax0n » Thu Aug 24, 2006 6:03 pm

67 files parsed. 0 new networks today. After 12 hours?
With the new algorithms, it is highly possible that there are hundreds of "cleaned up" datasets that are being re-processed which contain old data, therefore, no new networks, just better data for those networks.

Postby whitedice » Thu Aug 24, 2006 6:08 pm

Does that mean the triangulation portion of the parsing is what is taking so long as it reconfigures where existing detected networks are (in so many words) repositioned with better data?

Postby mahlerrd » Thu Aug 24, 2006 6:14 pm

Hey, you guys could create a queue of delayed items and a watchdog process on the regular queue - whenever the watchdog process notices there's no items waiting to be processed, it could inject one of the delayed items.

Then, punch a bunch of those backlog jobs into the delayed queue and off you go.

Of course, the presumptions and assumptions are many in the above, so if it bears great similarities to the ramblings of a lunatic, it's because...

Oh wait, I don't think I ought to complete that thought.

Postby ax0n » Thu Aug 24, 2006 7:59 pm

Well, I haven't seen the WiGLE Queue get actually clogged to the point of no return since 2004 or so. Since this massive data dump is probably a one-time thing, it probably doesn't make sense to burn a lot of effort on making a high-priority and low priority queue system.

I've got 3 or 4 in line ready to process as well, but points are points no matter if they take 3 seconds or 3 days to register with the system.

Patience, grasshopper.

Postby NetBandit » Thu Aug 24, 2006 9:28 pm

I guess now would be a good time to remind people that they can combine their stumbles into a single upload file (merge files in netstumbler) so that the processing time can be minimized.

-NB

Postby NetBandit » Thu Aug 24, 2006 10:13 pm

OK, this pretty much explains it:

1 hratch 581,795 (8.336%) 904,806 52,897 17-Jul-2002 24-Aug-2006

11 whitedice 63,137 (0.904%) 87,872 14,624 30-Apr-2006 22-Aug-2006


-NB

Postby uhtu » Thu Aug 24, 2006 11:25 pm

heh. you folks are hilarious.

an extremely large ammount of data hit the site at the same time, which causes backups, which cause backups, which cause recursion :-)

the servers are up, and running as fast as their little hearts (and the hamsters-on-wheels that power them) can go. please be aware that because of some of the mechanics of the updates processing and reporting, what's *actually* been done, and what you see on the stats page will diverge until the job is done.

rest assured that we will bring the might and terror of our fully operational PokingSticks^tm down upon both the interns and the hamsters, until all the files are processed.

no byte left uncollated.

in the interim: patience, young padwans.

Postby Dutch » Fri Aug 25, 2006 8:58 am


rest assured that we will bring the might and terror of our fully operational PokingSticks^tm down upon both the interns and the hamsters, until all the files are processed.
So you're saying Monica is into the PokingStick^sm scene ?? That brings a whole new angle to the Monicagate scandal :shock:

Dutch
[url=http://www.wigle.net/gps/gps/StatGroup/listusers?groupid=20041206-00006][img]http://home19.inet.tele.dk/dutch/netstumblerwigle.gif[/img][/url]

Postby theprez98 » Thu Nov 30, 2006 1:22 am


rest assured that we will bring the might and terror of our fully operational PokingSticks^tm down upon both the interns and the hamsters, until all the files are processed.
So you're saying Monica is into the PokingStick^sm scene ?? That brings a whole new angle to the Monicagate scandal :shock:

Dutch
Did you break WiGLE again? I finally get my files uploaded, and now they're stacking up... :P

Postby y2k98 » Mon Jan 29, 2007 7:02 am

it has been stuck at 20 all night, is everyhting ok?

15 posts • Page 1 of 1

Return to “WiGLE News”

Who is online

Users browsing this forum: No registered users and 30 guests