AirNav Systems Forum

AirNav RadarBox and RadarBox24.com => AirNav RadarBox and RadarBox24.com Discussion => Topic started by: AirNav Development on April 13, 2010, 01:29:12 AM

Title: New Server Database Populate Routine
Post by: AirNav Development on April 13, 2010, 01:29:12 AM
We have just deployed some new database update routines on our server.

From now on the algorithm being used to retrieve data is the following:

1- Server is queried by mode-s;
2- If data exists and is less than 15 days old it is retrieved directly from AirNav server;
3- If it doesn't exist or is older than 15 days we query GAS databases by mode-s;
4- From GAS we get all aircraft details (no more details from airliners.net);
5- Using the registration we get from GAS we then query airliners.net by registration and ordered from the most recent to the older photo (thus getting the 2 most recent photos);
6- GAS aircraft aircraft information and airliners.net photos are retrieved to the user;

Our server database has been totally reset so new fields are just being added (2143 in the first 10 minutes). We will keep monitoring how the server performs under these new conditions.

As usual any questions/suggestions are welcome.
Title: Re: New Server Database Populate Routine
Post by: DaveReid on April 13, 2010, 07:20:20 AM
As usual any questions/suggestions are welcome.

Could you explain a bit more about how these changes will propagate ?

For example I've got N148AA in my log at the moment, which GAS shows correctly as a B350, but I'm still getting details and photos of a long since scrapped Hawaiian DC-10 in RadarBox.

Is there something I need to do at this end to refresh the data, or is it just a question of waiting a few hours for the server to update ?
Title: Re: New Server Database Populate Routine
Post by: tarbat on April 13, 2010, 07:26:12 AM
Great news!!  I'm guessing that point 5 means that photos of the wrong aircraft may still be retrieved for "unusual" registrations - eg. German Air Force?

Dave, I'm guessing that if the end-user wants to see these updates, they will need to empty out their aircrafts table in navdata.  And then run Populate in MyLog.
Title: Re: New Server Database Populate Routine
Post by: bratters on April 13, 2010, 07:46:33 AM

Dave, I'm guessing that if the end-user wants to see these updates, they will need to empty out their aircrafts table in navdata.  And then run Populate in MyLog.

Tarbat - if you, Dave or Airnav can confirm the above-mentioned course of action for us non-techies, it would be greatly appreciated.

Sounds promising.

Title: Re: New Server Database Populate Routine
Post by: DaveReid on April 13, 2010, 07:48:21 AM
Dave, I'm guessing that if the end-user wants to see these updates, they will need to empty out their aircrafts table in navdata.  And then run Populate in MyLog.

OK, though I was rather hoping for an answer that didn't include the word "guess" :-)

If what you say is correct, it begs the question as to whether emptying the NavData aircraft table is within the capability of the average, non-SQL-knowledgeable RadarBox user.

AirNav ?
Title: Re: New Server Database Populate Routine
Post by: AirNav Support on April 13, 2010, 07:57:36 AM
The changes are to ensure more reliable data is being pumped through to the databases.

Obviously the database on your local machine is a static, and a software update will be needed to ensure changes are picked up or a refresh proceadure added.

This is the first stage, so we can ensure reliable data is being passed through and server load is fine before updating the software.

However yes, a clean of your NavData database would force it to get fresh data from the server for all aircraft. For customers who don't do this only new aircraft you pick up will be checked using the new routines.
Title: Re: New Server Database Populate Routine
Post by: AirNav Support on April 13, 2010, 08:01:51 AM
Just as FYI:

The next stages will be to have a database which has a update/refresh proceadure and also allow the certain rows to be marked so they are not updated or refreshed, incase a customer is adding data manually that is not in our central database.
Title: Re: New Server Database Populate Routine
Post by: vonsmalhausen on April 13, 2010, 08:14:14 AM
Thanks Airnav, this sounds verry promessing and is what most of us just want, i hope this update can be expected soon.
in the meanwhile, is there a kind soul who can give me ( and i'm shure others ) a step by step guide for dummies how to clean up that NavData db ; i don't want to mess up my system by trying to refresh my data
Title: Re: New Server Database Populate Routine
Post by: DaveReid on April 13, 2010, 08:28:25 AM
For example I've got N148AA in my log at the moment, which GAS shows correctly as a B350, but I'm still getting details and photos of a long since scrapped Hawaiian DC-10 in RadarBox.

Is there something I need to do at this end to refresh the data, or is it just a question of waiting a few hours for the server to update ?

I can confirm, for the example I quoted, that deleting the record from the NavData aircraft table and then refreshing from the server now brings up details of the Super King Air (although the ICAO code is the dreaded three dots).

The photos still show the DC-10, though - how do I fix that ?
Title: Re: New Server Database Populate Routine
Post by: bratters on April 13, 2010, 08:31:28 AM
Thanks Airnav, this sounds verry promessing and is what most of us just want, i hope this update can be expected soon.
in the meanwhile, is there a kind soul who can give me ( and i'm shure others ) a step by step guide for dummies how to clean up that NavData db ; i don't want to mess up my system by trying to refresh my data

You took the words out of my mouth.
Title: Re: New Server Database Populate Routine
Post by: AirNav Support on April 13, 2010, 08:31:52 AM
Clear out the photo from photo folder and see whether that helps. If not check the registration in airliners.net and see whether it does have a a photo of the B350.

This is still an issue that is being worked on, that if the photo and and aircraft details don't match, whether to show any photo at all.
Title: Re: New Server Database Populate Routine
Post by: vonsmalhausen on April 13, 2010, 08:35:59 AM
btw, i'm not getting contact with the server right now, so no network or 3D
are there troubles with the server, or is it a problem on my side ?
Title: Re: New Server Database Populate Routine
Post by: Runway 31 on April 13, 2010, 08:37:08 AM
network down
Title: Re: New Server Database Populate Routine
Post by: phil zech on April 13, 2010, 08:41:15 AM
Same here,  I am also getting in My Flights , bottom right ,45 Processing Photo/Info Requests...

and increasing as I get more aircraft in My Flights. I assume this is connected to server problems.

Regards

Phil
Title: Re: New Server Database Populate Routine
Post by: AirNav Support on April 13, 2010, 08:43:21 AM
We are investigating, this is also why we are not providing a way to clean your navdata database at the present time as we want to monitor server load as we know refresh counts etc.. will put an additional strain on the server.

Please bear with us.
Title: Re: New Server Database Populate Routine
Post by: [email protected] on April 13, 2010, 08:56:44 AM
having difficulty with network just wondered if server was offline as I cannot connect up to it
Title: Re: New Server Database Populate Routine
Post by: jannuh on April 13, 2010, 09:21:07 AM
Thanks Airnav, this sounds verry promessing and is what most of us just want, i hope this update can be expected soon.
in the meanwhile, is there a kind soul who can give me ( and i'm shure others ) a step by step guide for dummies how to clean up that NavData db ; i don't want to mess up my system by trying to refresh my data
Is it a good idea to start all over again, with db and photos??

Did a new start with V3.13, Dec. 10th, got 13.927 planes in Mylog again, so within a few months I could get more up to date data...
Title: Re: New Server Database Populate Routine
Post by: abrad41 on April 13, 2010, 02:26:29 PM
Quote
The next stages will be to have a database which has a update/refresh proceadure and also allow the certain rows to be marked so they are not updated or refreshed, incase a customer is adding data manually that is not in our central database.

I think I shell be waiting until the above is complete before doing anything with my Navdata file, I have done a lot of manual updating of my database.

Even GAS don't have all the details of everything, I use a combination of GAS, Airframes etc etc to get all of my information.

Airnav - Can I just say (From the statment above), there is no way I am knocking what you are doing, you are moving things in the right direction - Thank you for that. I just find the combination of all the sites helps me.

Andy
Title: Re: New Server Database Populate Routine
Post by: tarbat on April 13, 2010, 02:38:01 PM
OK, though I was rather hoping for an answer that didn't include the word "guess" :-)

Sorry that wasn't good enough for you Dave, but I don't have any inside information from Airnav, so a "guess" is the best I could do.
Title: Re: New Server Database Populate Routine
Post by: gleff on April 13, 2010, 03:23:24 PM
Just by curiosity, where do the routes come from?

Say an aircraft comes into view that isn't in the database.  It gets populated by the GAS data.  Is the routes a manual job, or does that come from GAS as well?
Title: Re: New Server Database Populate Routine
Post by: tarbat on April 13, 2010, 03:40:01 PM
I think (yes, a guess), that routes come from http://www.flightstats.com
Title: Re: New Server Database Populate Routine
Post by: gleff on April 13, 2010, 03:43:42 PM
I think (yes, a guess), that routes come from http://www.flightstats.com

So does that mean it's something I have to do manually, or does the radarbox software automatically get that at the same time it gets the other info from GAS?

I'm assuming it's a manual job.

Title: Re: New Server Database Populate Routine
Post by: tarbat on April 13, 2010, 03:51:19 PM
No, it's NOT a manual job.  Routes are automatically retrieved from the Airnav servers, and I think they get the routes from flightstats.

You can see what route data is retrieved using the instructions posted at http://radarspotters.eu/forum/index.php/topic,878.0.html
Title: Re: New Server Database Populate Routine
Post by: gleff on April 13, 2010, 03:55:24 PM
Ahhh.. ok.. thanks.

Airnav.. can you advise how up to date the routes are from your servers?  Eg.  Do you get the information from Flightstats or where ever you get it from once a day, once a year, every lookup??? 

I'm just trying to gauge how up to date they'll be so I can determine if I should go through them and update them manually.  The other day I saw a flight that was very wrong but I'm not sure if it was a one off thing or not.  I didn't take note of the details.
Title: Re: New Server Database Populate Routine
Post by: tarbat on April 14, 2010, 10:11:07 AM
This looks like a bug in the new server routine.  This aircraft has the REGISTRATION populated with the aircraft type.

(http://farm5.static.flickr.com/4031/4519865027_6f3f7e6871_o.jpg)

(http://farm5.static.flickr.com/4014/4520504112_666fafa1bc_o.jpg)

I then corrected this to the correct registration (ZG511) in Database Explorer, but Radarbox overwrote my changes again.
Title: Re: New Server Database Populate Routine
Post by: Runway 31 on April 14, 2010, 10:14:16 AM
I know how to amend data on Database Explorer but how do you add data.  When I note a new aircraft which inst on the database, if I click the add record button and nothing happens?
Title: Re: New Server Database Populate Routine
Post by: RodBearden on April 14, 2010, 10:20:27 AM
Runway 31

When you click Add, a new row is added at the bottom of the table - click on the table then press Ctrl-End, and you'll find it.

Rod
Title: Re: New Server Database Populate Routine
Post by: Runway 31 on April 14, 2010, 10:23:02 AM
Cheers Rod, I will have a look and try it out.

Thanks

Alan
Title: Re: New Server Database Populate Routine
Post by: abrad41 on April 14, 2010, 10:27:20 AM
Quote
This looks like a bug in the new server routine.  This aircraft has the REGISTRATION populated with the aircraft type.
Quote
I then corrected this to the correct registration (ZG511) in Database Explorer, but Radarbox overwrote my changes again.

Chris are you still downloading the photo's with this

Andy
Title: Re: New Server Database Populate Routine
Post by: tarbat on April 14, 2010, 10:37:01 AM
Andy, yes.  Just highlighting that the new server routine appears to be mixing up the AR and AN fields.
Title: Re: New Server Database Populate Routine
Post by: abrad41 on April 14, 2010, 10:43:03 AM
Ok Chris

Just wanted to confirm the reason for the overwriting, this one of the things that is top of my list.

Andy
Title: Re: New Server Database Populate Routine
Post by: Runway 31 on April 14, 2010, 10:56:05 AM
Rod, had a try, nothing doing, probably me not explaining my problem properly I expect

An example on network flight there is flight ID GOL1269 E4833A but it doesnt give the aircraft type, registration, silhouette etc.  If I right click asking to locate mode S code in database explorer, the explorer table remains empty.  Googleing the registration shows this aircraft to be PR-VBW B737-8EH .  Is there anyway of adding this information into database explorer or do I require to wait until the system updates itself?
Title: Re: New Server Database Populate Routine
Post by: RodBearden on April 14, 2010, 11:03:24 AM
Ah - if you've right clicked to get to Database Explorer, then there's a filter in operation - click the Show All button first, then click the Add Record button. If you add a new record with a filter in operation, the filter stops you seeing the added record.

I hope that's the answer, anyway.

Rod
Title: Re: New Server Database Populate Routine
Post by: Runway 31 on April 14, 2010, 11:06:14 AM
Cheers Rod, I will give that a try.  Many thanks.

Alan
Title: Re: New Server Database Populate Routine
Post by: Runway 31 on April 14, 2010, 11:24:04 AM
Thanks Rod, worked a treat. I appreciate your time, shows the benefit of the forum.

Alan
Title: Re: New Server Database Populate Routine
Post by: RodBearden on April 14, 2010, 11:26:20 AM
It's a pleasure :-)

Oh - and once you've finished updating the new record, remember to click on a different, existing record before you close the Explorer window - that confirms that you've finished adding the new record and gets it written to the database.

Rod
Title: Re: New Server Database Populate Routine
Post by: Runway 31 on April 14, 2010, 11:39:34 AM
Thanks Rod, noticed that, eventually.
Title: Re: New Server Database Populate Routine
Post by: gleff on April 16, 2010, 02:01:01 PM
I accidently deleted my MyLog last night but managed to still have my NavData.db3 file intact.  However, changes that were made previously to the database seem to have disappeared.  It may have been something I did.

Just a question.  When aircraft details are updated and added to the database, is it based on the details in the MyLog?

In other words, does it check the database for an existing record before updating it, or does it just check the MyLog and then update the database accordingly?

I added parts of a record manually in the database, then lost the MyLog, and then the next time the aircraft was seen, the record lacked the information I had changed.
Title: Re: New Server Database Populate Routine
Post by: DaveReid on April 16, 2010, 02:26:34 PM
In other words, does it check the database for an existing record before updating it, or does it just check the MyLog and then update the database accordingly?

I added parts of a record manually in the database, then lost the MyLog, and then the next time the aircraft was seen, the record lacked the information I had changed.

This post

www.airnavsystems.com/forum/index.php?topic=4691.msg46741#msg46741

from AirNav Support would suggest that the measures to ensure that your own updates aren't overwritten by older information from the server are not yet in place.  Presumably they will require a new release of RadarBox in order to be implemented.
Title: Re: New Server Database Populate Routine
Post by: gleff on April 16, 2010, 02:45:26 PM
oh crap.. Must have missed that post.. this means all the changes I just did will probably get overwritten again... double crap.


Title: Re: New Server Database Populate Routine
Post by: eggplant on April 16, 2010, 10:07:34 PM
AirNav/Dave,

I've read with interest the posts on this thread regarding changes to the AN database. I unticked my auto-populate box several months ago and have been making manual corrections since. Thanks to Dave for the incorrect hex mode references  - I hope they are allowed to continue (if not perhaps Dave could post them on another site forum ?).

My question is, can I now assume the AN database is atively up to date (of course I know there is no such thing as a perfect database), and therefore I can tick my auto-populate box and stop making manual updates ?

Thanks
 
Title: Re: New Server Database Populate Routine
Post by: eggplant on April 17, 2010, 04:54:44 PM
I'll take that as "no" then.

Thanks for your help.