AirNav Systems Forum

AirNav RadarBox and RadarBox24.com => AirNav RadarBox and RadarBox24.com Discussion => Topic started by: apjwright on September 26, 2008, 07:22:41 PM

Title: Database Update Question
Post by: apjwright on September 26, 2008, 07:22:41 PM
So, I've been noticing that there are a lot of aircraft that are not showing a registration on my RB 'myFlights' list that i feel should have a registration (i.e. they are not new aircraft and haven't changed operators/countries/registrations lately).

How can I get my database more current than it is?

Thanks,
Tony
Title: Re: Database Update Question
Post by: AirNav Support on September 26, 2008, 07:25:49 PM
Are you connected to the Internet when RadarBox is running as it should automatically populate the aircraft details?
Title: Re: Database Update Question
Post by: apjwright on September 27, 2008, 01:24:43 AM

yes ...

If I were to, say, delete the database, would it be re-created the next time I connect?
Title: Re: Database Update Question
Post by: Allocator on September 27, 2008, 08:17:46 AM

yes ...

If I were to, say, delete the database, would it be re-created the next time I connect?

Bad idea that - the aircraft details in MyFlights comes from the main NavData.db3 database, as do the routes, aircraft types etc.  The database is updated as each new aircraft is detected, it's not a "database update" as such, more like an aircraft details auto populate.

You do mean MyFlights and not MyLog?
Title: Re: Database Update Question
Post by: testmonkey on September 27, 2008, 10:32:41 AM
Tony,

If (as we suspect) you are talking about MyFlights i.e. live traffic that you are watching then as Allocator said deleting the database isn't the way to go. Anyway if you did delete the database that RB uses to populate these details, the NavData.db3 database, I would assume that your RB wouldn't start as that database is validated as part of the start-up procedure.

If you use the Database Browser within RB you'll get some idea of the structure of the database. The database is in SQLite format so you could download one of the free SQLite editors available on the net which is the best way to go if you intend doing lots of or continual updates. In theory you could delete all of the entries from the Aircraft table and let that repopulate but to be honest I wouldn't go down that route myself. I did delete 30 odd thousand records from my database that only had ModeS codes listed so that they can repopulate over time if I ever pick them up but not any others. Obviously you'll need to use or learn SQL to perform some tasks but if you search these forums you'll find a couple threads with some SQL scripts to get you started.

To keep the database up to date or populate the extra details i.e. reg for those that are missing you'll either need to do them manually or use something like sbspopulate to batch process them. If you search the Gatwick Aviation Society database (this is what RB uses but it never seems to be quite as current as going to the website direct) at
http://www.gatwickaviationsociety.org.uk/modeslookup.asp
or the airframe.org database at
http://www.airframes.org/
you can find known details to update from. Be aware that the airframes site limits searches to 100 per day.

In addition to missing registrations you'll also find switched construction number and company name fields, missing short type, company name, construction numbers and things like inconsistent company names (Easyjet, Easyjet Airlines, Easyjet Airline Company) which all need changing to get as much accuracy as possible. I've had my RB for a couple of months now and I'm still working through my Aircraft table to get it all sorted.

HTH
Title: Re: Database Update Question
Post by: Yoda on October 02, 2008, 09:54:27 AM
Test Monkey wrote: To keep the database up to date or populate the extra details i.e. reg for those that are missing you'll either need to do them manually or use something like sbspopulate to batch process them. If you search the Gatwick Aviation Society database (this is what RB uses but it never seems to be quite as current as going to the website direct) at
http://www.gatwickaviationsociety.org.uk/modeslookup.asp
or the airframe.org database at
http://www.airframes.org/
you can find known details to update from. Be aware that the airframes site limits searches to 100 per day.

Does this imply that the sbspopulate will work with the AirNav mode-s code table. Sorry, not that familiar with PC based tools.
Title: Re: Database Update Question
Post by: Allocator on October 02, 2008, 10:45:53 AM
No, RadarBox doesn't need SBS Populate as it has it's own built-in autopopulate function that gets the data from the GAS database via the AirNav server.
Title: Re: Database Update Question
Post by: tommyg on October 02, 2008, 01:57:53 PM
On the database question I notice in my flights that I still get either ... or null on aircraft that have been seen many times before despite my attempting to edit them in My Log. This happens regularly on CityJet RJ85's and Flybe E190's.
When I use the quick filter the picture comes up with correct type but the aircraft type is either of the above and never updates despite using autopopulate. Am I doing something wrong?
Title: Re: Database Update Question
Post by: Allocator on October 02, 2008, 02:48:47 PM
On the database question I notice in my flights that I still get either ... or null on aircraft that have been seen many times before despite my attempting to edit them in My Log. This happens regularly on CityJet RJ85's and Flybe E190's.
When I use the quick filter the picture comes up with correct type but the aircraft type is either of the above and never updates despite using autopopulate. Am I doing something wrong?

The MyLog database does not affect what you see in the MyFlights list.

There are 2 databases:

 - Navdata.db3 - this is the main source database that is autopopulated when new aircraft are detected.

- MyLog.db3 - this is a record of the live aircraft that have been picked up by your antenna.

Editing MyLog.db3 via the MyLog interface is only going to change details for your historic record of aircraft.  To change the details in the source database (NavData.db3) you must use the RB Database Explorer via the File menu.

See the FAQ section here on the forum for a guide to manually editing aircraft and route details.


Title: Re: Database Update Question
Post by: Yoda on October 02, 2008, 03:02:29 PM
No, RadarBox doesn't need SBS Populate as it has it's own built-in autopopulate function that gets the data from the GAS database via the AirNav server.

I appreciate that, and thats fine when the box is used where internet connection is available. What I am after is a way of updating the NavData.db3 without having to enter the information field by field using the provided edit tool.
Title: Re: Database Update Question
Post by: Allocator on October 02, 2008, 03:13:22 PM
No other way as far as I know.

There are various 3rd party SQL database editors, but they all require the fields to be edited separately.  There isn't a "data inport" as such, but if you are clever with SQL, then you might be able to do something.
Title: Re: Database Update Question
Post by: Yoda on October 03, 2008, 08:58:49 AM
Allocator - thanks. I suppose it was too much to hope sbspopulate would work. Looks like I'll have to continue to update the long way and revise my sql skills in the long term!
Title: Re: Database Update Question
Post by: Allocator on October 03, 2008, 10:02:57 AM
Yoda,

I find that I only have to do minimal manual update work, even when I'm using RB away from and Internet connection.  If you use the same PC, say a laptop, both with the connection and then without this works well because the main source database NavData.db3 is auto populated when you are on line and then these details are available when you are next working off-line.

Obviously, if you get a new aircraft when you are off line, then you won't see all the details for it.  Also, the records in MyLog - your detected aircraft - will have details missing.  Every now and then, I use Populate in the MyLog Tools menu, this looks at the NavData database and also checks the Server if you are on-line to fill in any missing details.

If you are using different PC's, then just copy the NavData.db3 file from the Internet machine to the portable machine, and your  source database will always be up to date.  I think that the only time you will have many aircraft without details if if you take your RB to a different continent.
 

Title: Re: Database Update Question
Post by: tarbat on October 03, 2008, 10:22:59 AM
One technique I have found useful for correcting <1000 aircraft at a time is to use the RePopulate function in GAS sbspopulate.  To do this, you'll need to get hold of an empty SBS-1 Basestation database.  You can then populate that database with the ModeS Hex Codes of the aircraft you want to correct.  Then run RePopulate in GAS.  You then need to export the aircraft from the Basestation database, and import them into your NavData database.

Very complicated, but easy once you've setup the various queries in SQLMaestro.  Great for correcting aircraft records where:
1. The Serial and Owner are reversed.
2. The ICAOType doesn't have a match in the silhouettes folder.
3. Empty registration numbers.
4. Etc.....
Title: Re: Database Update Question
Post by: DorsetGooner on October 03, 2008, 01:36:48 PM
I have a similar problem with all Delta Flights. They don't appear to show up on the map although using modern aircraft and I never have the airline logo show up in MyFlights. Is this an issue with Delta for everybody or just me ?

Regards
Dorset Gooner
Title: Re: Database Update Question
Post by: tommyg on October 03, 2008, 03:48:05 PM
Does anyone else have problems updating the database manually. I can get it to add data but when i try and move along the row to edit nothing happens. When i close the page down and go back into it there are just lots of blank pink coloured rows with no information. This is gettin very frustrating as nearly all my cityjet flights are still showing ... and will not take load as RJ85
Title: Re: Database Update Question
Post by: Hawkeye on October 03, 2008, 03:51:05 PM
DG,

Hardly any Delta flights transmit ADB-S info. Today for instance, N174DZ, N175DZ,N176DZ, N177DZ, N187DZ, N188DN, N1501P, N1604DZ, all B767s appeared in MyFlights without any callsigns or other information. Only N707TW, a B757 operating flight DAL045 was transmitting ADS-B. data. Other 757s in their fleet have also previously appeared in my logs with data but apart from a couple of aircraft in the N8** series, their B767s always remain 'secretive'

Anyone know why? 

Regards
Hawkeye
Title: Re: Database Update Question
Post by: Allocator on October 03, 2008, 04:28:12 PM
Does anyone else have problems updating the database manually. I can get it to add data but when i try and move along the row to edit nothing happens. When i close the page down and go back into it there are just lots of blank pink coloured rows with no information. This is gettin very frustrating as nearly all my cityjet flights are still showing ... and will not take load as RJ85

It sounds like you are trying to add a record when one already exists and just need updating.  The database only allows one instance on and Mode S hex code.

Say you want to add the details for

4CA6BB

Use File > Database Explorer then make sure you have selected the "aircraft" table in the drop-down at the top.

Type 4CA6BB in the "Text to find" box

Select "IN" in the Condition box

MS in the Field box (Mode S)

Click find

If a record exists, it will appear and you can edit it.

If no record exists, click "Show All" then click "Add Record" and you add will be OK.

Just try it with a Mode S that already exists, and you won't be able to move to the next field.

To get rid of the blank pink entries, just click in one of the fields in Database Explorer and click Delete Record.



Title: Re: Database Update Question
Post by: tommyg on October 04, 2008, 10:23:16 AM
Thanks Allocator, thats solved one issue. Just curious as to why it only affects certain acft types i.e. RJ85's and E190's in my case. All other types populate automatically and show on map correctly.
Title: Re: Database Update Question
Post by: Allocator on October 04, 2008, 12:55:24 PM
Tommyg,

For some reason, the RB database comes with the RJ85 aircraft details populated, except that the short aircraft type is set to "..." - It's the same for the Flybe DH8D's too.  THe problem is that the RB auto-populate function wont overwrite data if it's already there, just in case you have made a deliberate manual update, so it see the records with "..." as genuine records.

I suppose that if you deleted all the aircraft with a type "..." then they should auto-populate.  However, it would probably take as long to delete them as it would to manually enter the details.  I just do them as they appear in the list, so I'm only doing one or two at a time.
Title: Re: Database Update Question
Post by: jgrloit on December 14, 2008, 06:16:16 PM
I have today seen a problem - a German Air Force - new Aircraft to me was populated via the Internet as a Thai Navy aircraft.
GAF shows it as a CL60 regn 12+06
ModeS shown was 3F5D9C...   
I suspect that the RB Server database needs some cleaning and updating.
Any ideas if this is true?
Title: Re: Database Update Question
Post by: Allocator on December 14, 2008, 06:45:42 PM
I have today seen a problem - a German Air Force - new Aircraft to me was populated via the Internet as a Thai Navy aircraft.
GAF shows it as a CL60 regn 12+06
ModeS shown was 3F5D9C...  
I suspect that the RB Server database needs some cleaning and updating.
Any ideas if this is true?

When you say that the aircraft details were populated, are you talking about the photograph or the aircraft data line?

We've had problems before with these GAF aircraft with odd callsigns.  In my database I have:

MS       AR       AT
3F8606 12+06 CL61

and

3F5D9C 1206 PT3

Do a search for 12+06 on airframes.org and see what confusion this generates :-0

Beast thing in this case is to do a amnual update for this using Database Browser
Title: Re: Database Update Question
Post by: ACW367 on December 14, 2008, 06:50:31 PM
QUOTING JGRLOIT
I have today seen a problem - a German Air Force - new Aircraft to me was populated via the Internet as a Thai Navy aircraft.
GAF shows it as a CL60 regn 12+06
ModeS shown was 3F5D9C...   


This is not a problem with Radarbox and has been discussed before. The fault is with the way that the airliners.net search function works.  

German Military aircraft in airliners.net are shown without the iron cross. Therefore your CL60 will be shown as 1026.  When you put 10+26 into the airliners net search, it doesn't look for 1026 but looks for two search criteria 10 plus 26. Which is why it can't find them and comes up with random photos which have serials and construction numbers containing either 10 or 26.  

A manual update doesn't  work fully as it will still look for a photo and then change the data.

The way around it is to copy the pictures of regularly seen GAF fleets direct from airliners.net into your airnav photos file and rename them as 10+26 etc.
Title: Re: Database Update Question
Post by: jgrloit on December 14, 2008, 07:00:40 PM
ACW367  - you misunderstood the problem:-

The aircraft was seen by the Airnav unit and software.
Had not been seen before, so was not in the local DB.
The information received via the Internet, from, as I understand it, the RB Server showed the 'owner' as Thai Navy .....
When I looked at the Gatwick DB, due the the 'flight number' being GAF457 - it showed as a CL60 regn 12+06 etc.
I therefore suspect that the B Server DB needs cleaning.
I have seeen this aircraft twice today and deleted the record between sightings, hoping for a more correct ID the second time.
I have now had to manually correct this entry, and wonder how many more errors there are on the RB Server!!!
Title: Re: Database Update Question
Post by: Allocator on December 14, 2008, 07:13:54 PM
Unfortunately, no database is going to be 100 percent correct.  As I understand it, the AirNav server updates from the GAS server every now and then - I don't know how often it does this although I know that data you have in your local database will not be overwritten, just in case you have manually updated that record.

Did you delete the record for 3F5D9C using the Database Browser (not by editing MyLog which is only historical data)?
Title: Re: Database Update Question
Post by: jgrloit on December 14, 2008, 07:20:37 PM
Yes deleted the record through the database Explorer, with the hardware processing stopped.
I do not process network flights normally, so this feature was not running either.

I had hoped that the RB Server would check the GAS database, when a second request was received from a recently updated remote.
It looks as though there is no recheck marker in use.
As a cross-check is there any was to look at the RB Server database in a similar way to checking directly, online, with the GAS database?
Title: Re: Database Update Question
Post by: Brian on December 20, 2008, 12:13:14 AM
It's been asked before.  Haven't seen a answer from Support/AirNav.   How does the user know when the AirNav Server DB been updated with newer info ?.

There should be a page where it shows when stuff been updated.
If a Radio Scanning Database can do it.  AirNav Should be able to do it!
http://www.radioreference.com/apps/db/?tab=history&ctid=219

Unfortunately, no database is going to be 100 percent correct.  As I understand it, the AirNav server updates from the GAS server every now and then - I don't know how often it does this although I know that data you have in your local database will not be overwritten, just in case you have manually updated that record.

-Brian
RadarBox User Forum
http://radarspotters.eu/forum/
Title: Re: Database Update Question
Post by: AirNav Support on December 20, 2008, 12:24:08 AM
If the aircraft is not our server it looks up from GAS. It does also periodicly try and check each of aircraft in the server with gas, it does not do this all in one go as the amount of aircraft is massive and it would overload GAS server.

So the updating goes on all the time.
Title: Re: Database Update Question
Post by: Brian on December 20, 2008, 12:34:11 AM
AirNav support.

If it does updating all the time.  How come N241AM hasn't been updated for the new aircraft.  That aircraft been flying since August (per a photo I seen). and the AirNav DB still shows the old aircraft info.

I first reported it on this topic.
http://www.airnavsystems.com/forum/index.php?topic=1821.0

-Brian
RadarBox User Forum
http://radarspotters.eu/forum/
Title: Re: Database Update Question
Post by: jgrloit on December 20, 2008, 02:27:07 PM
I am still seeing problem with the validity of data from the RB Server database.
Today 342553, seen for the first time here, filled out from the RB Server, with a French registration - does the RB Server have any validation on the data received from GAT, or elsewhere?
Also 30296B filled out with an American Registration.
What is going on???
Title: Re: Database Update Question
Post by: tarbat on December 20, 2008, 02:33:59 PM
342553 retrieved from GAS is EC-KCP, a Airbus A330-343X, serial 833.

30296B can't have come from GAS, as it's not on GAS.  So, I wonder where Airnav are getting the details from?
Title: Re: Database Update Question
Post by: Allocator on December 20, 2008, 02:48:51 PM
I am still seeing problem with the validity of data from the RB Server database.
Today 342553, seen for the first time here, filled out from the RB Server, with a French registration - does the RB Server have any validation on the data received from GAT, or elsewhere?
Also 30296B filled out with an American Registration.
What is going on???

342553 is F-WWKO isn't it, or am I missing something here?  This is what I have in my RB NavData.db3 database.

It's the same on airframes.org

Edit: Ah, I see now, F-WWKO was the test registration before it became EC-KCP

EC-KCP    Airbus   A-330-343X   A333   833   L2J      342553   IWD [TY] Iberworld   2007   F-WWKO   2007-04-16            active


30296B on airframes.org show as a Mode S miscode

N261AV    Airbus   A-320-214   A320   1615   L2J      30296B      2001   F-WWDR               wrong ICAO24

Using incorrect ICAO24 address 30296B on mode-S, instead of A282C8. [2006-04-05, 2008-12-02][/]

Title: Re: Database Update Question
Post by: jgrloit on December 20, 2008, 02:56:53 PM
What I spot is a break in sequence, and a flag that seems not to match the Country prefix.
I understood that the Mode-S is country based, and therefore the ICAO country code should, for civil aircraft always match the country of registration.
The above delivery error should be avoided, by the use of generic delivery Mode-S codes, which are temporary use ONLY, and could ignored when adding to the databases.
Title: Re: Database Update Question
Post by: Allocator on December 20, 2008, 02:58:57 PM
Are you telling us this as a fact, or is it how you think that it should work?

This must be a task for the country of registration regulator to deal with surely?
Title: Re: Database Update Question
Post by: jgrloit on December 20, 2008, 03:00:02 PM
ON another similar matter the Flight ID, again ICAO originated could be matched to the current Operator, and possibly be shown in addition to the Owner, or leaser of the Aircraft.
I don't know if the software fixes should be implemented on the RB Servers, or more possibly, for display accuraty, on each RB received, computer combination!!!
Title: Re: Database Update Question
Post by: AirNav Support on December 20, 2008, 03:02:53 PM
Problem is you will always find exceptions to the rules there isn't a easy fix like that jgrloit.
Title: Re: Database Update Question
Post by: jgrloit on December 20, 2008, 03:05:22 PM
Well I remember seeming, in certain magazines, years ago mention of delivery registrations, which seemed to be similar to UK car trade plates - held by a dealer or manufacturer.
These permitted the delivery of military aircraft, for instance, on civilian registrations, which were removed and the aircraft allocated it's "REAL ID" after delivery.
IT had something at the time to do with insurance, and listed pilots, as I recall.
Title: Re: Database Update Question
Post by: jgrloit on December 20, 2008, 04:47:42 PM
I have just checked to ICAO rules from :-
www.kloth.net/radio/icao24alloc.php
and they seem to follow my expectations.
The Country code MUST follow the registration country.
Re-Registration means a new Mode-S code!!!!!
Temporary delivery ID's are permitted, with a specific Mode-s. - follow the links from a registration ID on Airframes.org then use the delivery Reg as a source ID and see what I mean!!!
Title: Re: Database Update Question
Post by: derringer on December 20, 2008, 05:10:37 PM
hi
I noticed this problem : sometimes, when RB detect a military aircraft, the image doesn't match with the plane (it's normal in a sense),  so if I populate the database with correct data, thinking all is ok.
But, when the aircraft appears again, the wrong data take place again instead of my good ones.
So, my question are:
- does this bug fixed by rebooting the software?
- is there any way to populate the database correctly, as a little software?

Again, sorry for bad english translation :/ hope You understand all.
Title: Re: Database Update Question
Post by: Allocator on December 20, 2008, 05:26:08 PM
hi
I noticed this problem : sometimes, when RB detect a military aircraft, the image doesn't match with the plane (it's normal in a sense),  so if I populate the database with correct data, thinking all is ok.
But, when the aircraft appears again, the wrong data take place again instead of my good ones.
So, my question are:
- does this bug fixed by rebooting the software?
- is there any way to populate the database correctly, as a little software?

Again, sorry for bad english translation :/ hope You understand all.

I suspect that you are making the changes using the MyLog edititor and not Database Explorer.

Records manually edited using Database explorer will not be overwritten by autopopulate.

MyLog is only a record of what you have detected, it does not affect the data you see in MyFlights and Network lists.
Title: Re: Database Update Question
Post by: derringer on December 20, 2008, 06:46:27 PM
nice,thanks for the help ;)
Title: Re: Database Update Question
Post by: jgrloit on December 21, 2008, 03:03:19 PM
Can anyone comfirm, if once the Mode-S entry is in the local database, having been autopopulated by the system, are there ANY conditions under which the autopopulate is re-processed?
I am specifically wondering about the times when the tie-up has been extracted from the country sequence, or when there is an invalid aircraft type in the database.
Title: Re: Database Update Question
Post by: jgrloit on December 21, 2008, 04:02:19 PM
Since my last post, I have been considering the putting the following proposal to AirNav Development:-
When the Aircraft is Autopopulated, any the Mode-S to Registration is from the tie-up, ie no Aircraft type is known;
That the aircraft type (AN) is set to null;
When an aircraft is seen again and the local database field AN contains null a further update is attempted,  if more than 7 days have passed since the last attempt to update that aircraft.
This would mean that there would be a low volume of inter database traffic, but  eventually the full details would populate the local Database.

Any more takers??
Title: Re: Database Update Question
Post by: mike L on February 11, 2009, 11:55:56 AM
Hi
I can update the data base with new company info but then when its updated
by airnav in the backgrond it overrights the company info with untitled so
I can not update my local database can airnav filter out filds with untitled in tham

thanks mike
Title: Re: Database Update Question
Post by: mike L on February 11, 2009, 12:20:56 PM
Hi re my last post I now have a database with a lot of untitled and null in it
what to do next
Title: Re: Database Update Question
Post by: tarbat on February 11, 2009, 01:49:46 PM
Mike, when you say database, do you mean the MyLog, or the Navdata (what you see in Database Explorer)?  And which of these two databases did you manually update?
Title: Re: Database Update Question
Post by: mike L on February 11, 2009, 02:10:57 PM
Hi
tarbat 
I mean navdata the one u change with database explorer
I changed the airline/company info with up to date info then showall
then reclick the process hardware fflights my new info is there then is
overriten by Untitled am i right in thinking that if on new data was sent
it would not overwrite my new data
Title: Re: Database Update Question
Post by: jgrloit on February 14, 2009, 12:29:37 PM
A Database question, which is not Update related:-
Is the ACtype table actually used in RB, or is it a hangover from previous versions?
It seems that the data is actually held in full in the Aircraft table!!!

Does anyone have a table of Flight Numbers to Flight ID's - those used by Airports to those used on ADS-B Info?
Title: Re: Database Update Question
Post by: apjwright on September 07, 2009, 12:10:42 PM
One technique I have found useful for correcting <1000 aircraft at a time is to use the RePopulate function in GAS sbspopulate.  To do this, you'll need to get hold of an empty SBS-1 Basestation database.  You can then populate that database with the ModeS Hex Codes of the aircraft you want to correct.  Then run RePopulate in GAS.  You then need to export the aircraft from the Basestation database, and import them into your NavData database.

Very complicated, but easy once you've setup the various queries in SQLMaestro.  Great for correcting aircraft records where:
1. The Serial and Owner are reversed.
2. The ICAOType doesn't have a match in the silhouettes folder.
3. Empty registration numbers.
4. Etc.....

Today, I've been trying to research how to eradicate the dreaded "..." in the aircraft type field, etc. and generally improve the quality of data in my database.  As we all know, updating data using the Database Explorer is painful at best.  I found the post from Tarbat, above, but wondering how you actually put this method into practice?  How do I get an empty SBS database?  What are these SQLMaestro queries that Tarbat speaks of?  Seems like a lovely process, if you've got all the necessary bits-and-bobs.

Any help/guidance appreciated.

Tony - YYZ (currently perched comfortably in the Sofitel at LGW).
Title: Re: Database Update Question
Post by: GlynH on September 07, 2009, 05:56:58 PM
Hi Tony,

I asked the same question here;

http://www.airnavsystems.com/forum/index.php?topic=3245.msg30876#msg30876

but didn't see a reply...

I am sure I asked it somewhere else also...

I purchased SQLMaestro especially to do this (and other things) but I have not used it yet so I too would be interested in finding out the best way to remove all occurrences of "..."

Anyone? Tarbat??

Thanks & regards,
-=Glyn=-

Title: Re: Database Update Question
Post by: tarbat on September 07, 2009, 06:31:48 PM
Anyone? Tarbat??

I'll try to find some time this week to write-up some instructions.
Title: Re: Database Update Question
Post by: apjwright on September 07, 2009, 07:39:21 PM
That would be great - thanks very much.

Tony
Title: Re: Database Update Question
Post by: tarbat on September 08, 2009, 10:24:01 AM
Okay, I've had a quick go at documenting the process I follow to correct all the "..." in Aircraft Type (AT) field.  Some warnings:

1. It looks complicated, but once you've got the queries setup, it's a quick 15-minute job each week to correct the "..." and other data.

2. If you corrupt your database, don't blame me.  Take a backup BEFORE doing anything.

3. SBS Populate can't cope with more than 1,000 (I think) aircraft at a time.

4. Once you've got the hang of it, you can use the same process to correct other data, such as the "Untitled" company names, etc.

The instructions - http://www.tarbat.gofreeserve.com/instructions.zip
An empty Basestation database - http://www.tarbat.gofreeserve.com/basestation.zip
Title: Re: Database Update Question
Post by: apjwright on September 08, 2009, 10:27:21 AM

Thanks - will give it a try over the next few days ...

Tony
Title: Re: Database Update Question
Post by: EK01 on September 08, 2009, 01:29:23 PM
Okay, I've had a quick go at documenting the process I follow to correct all the "..." in Aircraft Type (AT) field.  Some warnings:

1. It looks complicated, but once you've got the queries setup, it's a quick 15-minute job each week to correct the "..." and other data.

2. If you corrupt your database, don't blame me.  Take a backup BEFORE doing anything.

3. SBS Populate can't cope with more than 1,000 (I think) aircraft at a time.

4. Once you've got the hang of it, you can use the same process to correct other data, such as the "Untitled" company names, etc.

The instructions - http://www.tarbat.gofreeserve.com/instructions.zip
An empty Basestation database - http://www.tarbat.gofreeserve.com/basestation.zip
URGENT !
Tarbat,

Don't know if it is just me but having clicked on your 2 sites in the above post, I was directed to completely different websites with a message coming up stating my machine had been infected seriously by a Trojan and in order to fix it to download and run a program which came up. Needless to say, I didn't but it took several goes to get out of the system. Having managed that, I shut everything down and I did a full system scan using Norton and nothing out of the ordinary was detected. I suspect anyone who goes ahead and downloads the program shown could have a serious problem. As I say, don't know if it is just me but perhaps you would be better deleting the post as it looks as if it could have been hacked.

Cheers,
Ian
Title: Re: Database Update Question
Post by: tarbat on September 08, 2009, 01:36:02 PM
Strange.  This is what I get when I download from my links:

(http://farm4.static.flickr.com/3421/3899706125_3b07c8a4b7_o.jpg)

Try http://www.tarbat.gofreeserve.com/instructions.htm

And I've attached the empty basestation database to this message.
Title: Re: Database Update Question
Post by: Fenris on September 08, 2009, 01:55:03 PM
Both links downloaded perfectly normally for me. I don't think it's a generic problem, probably specific to EK01.

Title: Re: Database Update Question
Post by: EK01 on September 08, 2009, 02:00:05 PM
Thanks for the replies, guys. I think I'll wait a bit before attempting again to see if anyone else has the problem. As I say, it could have been a hacker trying to get into my system but unsuccesfully due to running Norton and seeing if I would be naive enough to download their program.

Cheers,

Ian
Title: Re: Database Update Question
Post by: nortonbeak on September 08, 2009, 02:16:07 PM
Both links downloaded perfectly normally for me. I don't think it's a generic problem, probably specific to EK01.

I got a Trojan warning as well. I use Opera as my browser. As EK01, I was also re-directed.

Changing to Internet Explorer, I managed to get the download OK.
Title: Re: Database Update Question
Post by: Fenris on September 08, 2009, 02:36:25 PM
In my case, Firefox 3.5.2 plus NoScript worked without comment for both links.

It might be that NoScript prevented the execution of a script diverting to malware, that's one of the reasons I use it.
Title: Re: Database Update Question
Post by: 54901 - Jim on September 08, 2009, 03:02:40 PM
I also got the Trojan warning, but w/ Chrome.  I was able to successfully download the files w/ IE8.

Jim
Title: Re: Database Update Question
Post by: tarbat on September 08, 2009, 03:36:29 PM
Okay, I see this redirection happening in Firefox - to http://my-php.net/notify/2.php

But only if I click on the links directly.  If instead I copy the text of the link, then paste the URL in the address bar directly, then I don't see the redirection.

Sorry for this - I have no idea why gofreeserve does this :(  Shame there's a 300k limit on attachments in this forum.
Title: Re: Database Update Question
Post by: EK01 on September 08, 2009, 06:20:00 PM
Tarbat,

It was really just a general warning to perhaps some younger members or those not so computer savvy that if they get this 'download me' program coming up in order to 'fix' the trojan, NOT to do so but come out of the system and perhaps just double-check with their own anti-virus that nothing has slipped through.

Ian
Title: Re: Database Update Question
Post by: apjwright on September 10, 2009, 03:35:58 PM

I've successfully done all steps, except the last step to merge the updated SBSPopulate data into the NavData.db3 file.  I'm using SQL-Lite Maestro v9.3.1.5 - I have a totally different set of screens for the 'Import data' step than your instructions indicate.  There are no 'Update/append' type options, just map the fields and 'Import' - which then results in an error. 

Any thoughts?
Tony
Title: Re: Database Update Question
Post by: tarbat on September 10, 2009, 04:21:22 PM
Tony, can you post screenshots of the import process.  I'm on v7.12 of SQLM.

What error does it display when you import?  And you must have Radarbox closed when you do the import, otherwise it holds a lock on the database.

EDIT: Looking at the current online help, I would guess there's something accessed from the MORE button to specify which field is the primary key (MS), and to tell SQLM to update existing records with the matching primary key.

http://www.sqlmaestro.com/products/sqlite/maestro/help/05_07_03_import_data_wizard_fields/
Title: Re: Database Update Question
Post by: apjwright on September 11, 2009, 07:05:45 AM
Here are the screen shots from the 'import' step at the end of Tarbat's process.

Tried to post this yesterday, but it looks like it didn't take - perhaps because of the attachment.  I've broken this post into 3 because the screen shots make it too large.

I have scoured the options, there is nothing that allows you to specify the primary key field in the incoming data

Thanks,
Tony

[attachment deleted by admin]
Title: Re: Database Update Question
Post by: apjwright on September 11, 2009, 07:06:10 AM
File #2
Title: Re: Database Update Question
Post by: apjwright on September 11, 2009, 07:06:39 AM
and, finally, the error message.
Title: Re: Database Update Question
Post by: tarbat on September 11, 2009, 07:17:36 AM
What options are there if you click the MORE button?

The error message indicates that it's trying to insert new records, whereas you want it to update existing records.
Title: Re: Database Update Question
Post by: apjwright on September 11, 2009, 07:22:48 AM
The MORE button only has options to save/load mapping definitions.  Nothing about options related to updating vs. inserting records - that's the issue - have looked everywhere for that type of setting and come up with nothing. 

I will keep investigating.

Tony
Title: Re: Database Update Question
Post by: tarbat on September 11, 2009, 07:35:34 AM
Some other ideas:
1. What does the Build map button do?
2. Can you right click on the MS row to define it as the primary key?
3. What is the "Replacements" column for?
Title: Re: Database Update Question
Post by: jgrloit on September 11, 2009, 11:12:11 AM
Airnav
   I understood that when the option to download photographs was turned OFF the system should NOT obtain ANY information from airliners.net  - Is this correct?
The reason for asking is that even with 3v01b and photographs turned off I seem to be getting data and having the main database updates with incorrect information from the wild photograph searches.
ie     database update for new aircraft - from GAS, with correct information, then a short time later an update from a second source, corrupting the valid data!!!
Title: Re: Database Update Question
Post by: abrad41 on December 22, 2009, 08:31:04 PM
apjwright

Quote
apjwright
New Member

Posts: 30


    Re: Database Update Question
« Reply #69 on: 11 September 2009, 07:22:48 » Quote 

--------------------------------------------------------------------------------
The MORE button only has options to save/load mapping definitions.  Nothing about options related to updating vs. inserting records - that's the issue - have looked everywhere for that type of setting and come up with nothing. 

I will keep investigating.

Tony

Tony did you have any luck with the above - I am having the same problem, although I can not get it to work in a XML file - I used the CSV option. Might be an idea to try SQlite, to see what they say.

Andy
Title: Re: Database Update Question
Post by: abrad41 on December 23, 2009, 06:57:56 PM
For the attn of apjwright and tarbat.

I sent a e-mail to support at SQL, this is my e-mail
Quote
Category: Help
> Server Version: SQLite Maestro
> Product Name: SQLite Maestro
> Product Version: 9.3.1.5
> Comments: I am trying to import some data into a database, also trying to
> replace some data fields and one of the fields has a unique key, I am
> getting a error message = "SQL Error column MS is not unique".
> I know in previous versions you could insert or replace, is there a way of
> this in this version.
>
> Thank you

And there reply
Quote
Hi Andy,

Thank you for your message.

The feature you mentioned did not work as expected so we decided to
(temporarily) deactivate it. Hope we'll reanimate it in the future versions.

As a workaround you could import data to a temporary table and then execute a REPLACE statement.

Best Regards,
Vadim Vinokur
SQL Maestro Group

So to do as per tarbat instructions, I think we would have to change to a earlier version or maybe tarbat would know how to work something out with the REPLACE statement mentioned in there e-mail.

Hope this helps

Andy