AirNav Systems Forum
AirNav RadarBox and RadarBox24.com => AirNav RadarBox and RadarBox24.com Discussion => Topic started by: AirNav Development on October 24, 2012, 09:38:40 PM
-
We would like to have the feedback of our users on the bug listed below:
"Data (callsigns, altitudes, squawks, routes, etc) being assigned to the wrong aircraft."
This bug is also discussed at:
http://www.airnavsystems.com/forum/index.php?topic=6532.msg73788#msg73788
This is caused by having a single bit error in the transmission/receiving process and can be avoided from reaching the flight grid using software. The same error happens with any ADS-B decoder.
We our ideas on how to prevent this from happening.
Basically we will only show on the Grid (actual valid and correct received aircraft) data that has been received from a single aircraft twice.
For example if we have:
hexcode1,flightID1
The aircraft will be put on hold until:
hexcode1,flightID1
is received again and at this time it will be shown.
As usually we would like to know your feedback on this one and f you agree with this approach.
-
Sounds sensible enough, however I guess we will only find out with roll out of bug patches after internal testing. At the moment circa 1% of flights shown on the MyLog reporter on any given day are affected by this bug (IE regularly over 30 out of 3000 recieved daily).
-
Personally I'd display the information, but perhaps at half intensity (as if it was greyed out) until you have verified it with a second transmission, when you can display it at full intensity.
-
I would preffer the following precedure:
- Airframe appears greyed-out after first reception*
- Airframe gets valid after third reception in a row (just to be sure one more)
- Time between two "in a row receptions" should be limited to about let's say 5 to 10 seconds or so.
- Airframe will be available on port 30003 after it is validated.
this may also work for airframes were no flight id was received at all. Many of my decoding errors in my list are without a received flight id.
* Or you may concider to use the globe that is momentary indicating the shared ADS-B flights :
- dark-grey shaded globe: timed out airframes or any other that do not match up with one of the following:
- red (or dark yellow for the fellows with red/green problem): airframe awaiting validation
- green: validated non ads-b airframe
- blue(existing one): validated ads-b airframe
If adding all received airframes to MyLog, you should make it possible to indify the not validated ones, maybe by adding a column named valid and adding a y in it if it was validated. Just like you already do with the ADS-B status.
About the reports: You may put a + for non validated airframes were you already put a * for first time received airframes.
-
Maybe I am seeing this the wrong way and being a bit simplistic, but I wonder if this can even be called a bug. Spurious packets/fragments of data are flying about all over the place in between valid data and affect all receivers but are handled differently by each. From my point of view the box is trying to assign an identity to these mangled data fragments and is therefore assigning an identity to an aircraft which may not even be in the receiving area.
If this is the case why do we even want to see these mangled fragments/single packets being displayed on screen if they are not real identities. I therefore consider that these should be filtered out using software and are only displayed when they are considered valid. Software filtering should then minimise the number of mis-identified packets and we are only shown valid data. You will never eliminate mangled packets but will minimise their appearance on screen/logs.
Alan
-
As far as I know the ADS-B messages contain a 24bit parity check field which can be used to check whether the transmission is ok or there are errors.
I don't know the exact details yet, but as the parity check field is pretty big, I assume that it can even be used to restore received erraneous data. But the simple solution is to just ignore messages that are identified to have a transmission error.
Some links with more detailed information about this:
https://trainingzone.eurocontrol.int/ATMTraining/PreCourse/SUR/ADS/Taste%20the%20Course/32501.10.32657.85.28722/Default.html (https://trainingzone.eurocontrol.int/ATMTraining/PreCourse/SUR/ADS/Taste%20the%20Course/32501.10.32657.85.28722/Default.html)
http://www.radarspotters.eu/forum/index.php?topic=5617.0 (http://www.radarspotters.eu/forum/index.php?topic=5617.0)
http://jetvision.de/sbs/adsb/crc.htm (http://jetvision.de/sbs/adsb/crc.htm)
And Google returns a lot more:
https://www.google.com.au/search?hl=en&safe=off&q=adsb+parity+check&oq=adsb+parity+check&gs_l=serp.12...0.0.0.127946.0.0.0.0.0.0.0.0..0.0...0.0...1c.duPabi4gY8Y (https://www.google.com.au/search?hl=en&safe=off&q=adsb+parity+check&oq=adsb+parity+check&gs_l=serp.12...0.0.0.127946.0.0.0.0.0.0.0.0..0.0...0.0...1c.duPabi4gY8Y)
So I think that the best solution is to use the parity check to identify transmission errors instead of waiting for second messages or other workarounds.
-
I am statistics lover. And for me personally this is a bug. And very, Very, VERY sensible bug :)
If you, AirNav, will correct it I am ready to close my eyes on the most other positions in the bug list. Even this problem will be solved partially and it would reduce the display of aircraft data on the edge of my reception range, as colleague tarbat correctly mentioned.
We have now perfect coverage of Moscow FIR in our network so I don’t worry about my own reception range at all. I worry about my statistics tables and daily reports which are now full of errors due to this bug.
Please validate correct combination twice or even more times, or use a more complicated procedure but give me clear daily reports :)
The other problem for me here is “U letter” problems in FlightID. It’s mostly connected to cargo flights. The same flight displayed in my log twice with the different FlightID’s – let say ABW243 and ABW243U. Even this is not a software error, we need to do something with this issue.
Regards,
Nick
-
Nick,
I would think the flight ID problem is coming from the equipment on the aircraft itself and nothing to do with Airnav software. I have on occasion seen maybe over 20 different flight ID's in a very short space of time from the one aircraft, how would the receiver be able to identify which if any of these flight Id's is the correct one.
Alan
-
If this is the case why do we even want to see these mangled fragments/single packets being displayed on screen if they are not real identities. I therefore consider that these should be filtered out using software and are only displayed when they are considered valid. Software filtering should then minimise the number of mis-identified packets and we are only shown valid data.
Totally agree. I've recommended to AirNav the proposed approach as part of the v4.04 development work. Only when two (or three?) messages with the same hex code and data (squawk, FlightID, altitude, etc) have been received is the data considered valid and NOT from a mangled packet. This should prevent most corrupted data - unless you're so unlucky to receive two (or three?) messages with exactly the same mangled bits! I did suggest they open this up to the forum in case anyone could see any flaws in my suggested approach.
My opinion is that TWO identical messages is enough to validate the data. As you increase this to THREE or FOUR, you risk losing useful data.
-
As I wrote before, there is a proper way of handling the messages so that corrupted messages can be identified and maybe even corrected. The parity check is a standard algorithm that every programmer should be familiar with. It shouldn't be too hard for Airnav to implement it.
Then you do not need to wait for any messages and will always have correct information and not just decreased the likelihood of wrong data to be displayed.
-
As I wrote before, there is a proper way of handling the messages so that corrupted messages can be identified and maybe even corrected. The parity check is a standard algorithm that every programmer should be familiar with. It shouldn't be too hard for Airnav to implement it.
Then you do not need to wait for any messages and will always have correct information and not just decreased the likelihood of wrong data to be displayed.
As far as I know the functionality of simple parity checks are usually directly implemented in the Hardware. If there is the possibility of error correction in the "parity Data", then it would be part of software to use this error correction Block to re-calculate the original data. Because of the fact that this are usually quite simple standard algorithms too, this is often directly implemented in the hardware too. The question is, are the ADS-B decoding part of the Box equipped with this kind of error handling or not. If yes: your proposal won't work. If not, and only the raw data is sent to software, then you are right and they shall integrate the error correction routines to the software.
But remeber there is also the possibilty that the scramble adds up to a valid parity check, although the data is corrupted. Then a THREE (My own programming experiency told me, that TWO are not enough) needed receptions before valid will filter these ones out.
-
Not so sure about the proposed solution to this problem being to show information on the grid after only two or three messages with the same hex code have been received.
Have a look at the attached piccy, MyLog.jpg
Last Sunday, the 21st, I picked up Shuttle 7P from EGPF to EGLL with the code 40105C, correctly relating to G-AVGA, a Piper24 which was showing as operating this flight in My Log
You will see that it was being logged from 1546 to 1605 with a message count of 111. With so many messages over a period of 19 minutes it must surely have transmitted its code more than two or three times.
When AirNav opened this thread I dug a bit deeper into this flight and took a look at:- http://www.flightradar24.com/data/flights/ba1489/
BA1489 is the flight number for SHT7P. I found that the ‘plane actually operating SHT7P on the 21st was of course an A321, G-EUXL code 4010DC, - one character different. If you take a look at the above site you’ll see that at 1555, SHT7P was overhead Blackpool, only thirty miles from my home in East Lancashire. As MyLog shows it was between levels 27000 and 32000+, well within reception range, (100 miles approx in that direction), so the false data was not caused by the flight being at the extreme edge of my coverage.
My first thoughts were that the aircraft must have been continuously transmitting an incorrect hex code and the glitch was not in the AirNav software because, on checking today, not once in the 19 minute period did G-EUXL appear in the log, But it would appear the plane must have been sending correct info or the data produced at the above site would have been wrong too.
One more point on this problem which has been in my mind me for quite some time.
If one hex code character difference can cause so much confusion in a flight at such close range, just how often does it happen without us realising it?
I wonder if RB owners who use it for spotting, regularly log the wrong plane where the data isn’t so obviously incorrect as the SHT7P one
For example, BA A319s G-EUPE and G-EUPF have the codes 40083B and 40083C.
If one of these was affected by ‘the bug’, unless the user actually read off the registration, at an airport for instance, I very much doubt they would question the validity of the registration data produced in MyLog if they had seen it as an overflight from their window.
Regards
Syd
-
Tks for all your input.
"but remeber there is also the possibilty that the scramble adds up to a valid parity check, although the data is corrupted."
This is an error that cannot be avoided by any hardware and the reason to implement further software checks.
>Then a THREE (My own programming experiency told me, that TWO are not enough) needed receptions before valid will filter these ones out.
We will implement this to the mode-s, callsign and squawnk fields.
All the other fields don't need this since they may constantly change (altitude, groundspeed, track, verticalRate, airspeed, latitude, longitude) althought other checks are already being done on these fields (purple rain problem, etc).
Please confirm you agre with our approach and the coding will start.
-
I havent a clue to the problem here Syd and whether the proposed solution would work ot not, far too techy for me. G-EUXL flew the Glasgow-London route all day on 21st. As well as SHT7P it also flew SHT6R, SHT8K and SHT9B, did you pick it up operating them and if so what was its hex used on those flights.
I would however suggest implements the proposal.
Alan
-
I agree with your approach and will be impatiently waiting for final results.
Nick
-
I agree with your solution.
Niklas
-
This is now fully implemented.
Mode-s Hex code, Squawk and Callsign will only be accepted when received fro the third time.
We are working on other issues right now.
-
That's quick for fixing one bug.. Hope to see the 4.04 soon.
Regards,
-
I havent a clue to the problem here Syd and whether the proposed solution would work ot not, far too techy for me. G-EUXL flew the Glasgow-London route all day on 21st. As well as SHT7P it also flew SHT6R, SHT8K and SHT9B, did you pick it up operating them and if so what was its hex used on those flights.
I would however suggest implements the proposal.
Alan
Too techy for me as well Alan.
I checked MyLog for the 21st again and have no entries for 6R, 8K or 9B, but………
I decided to run a Registration filter for G-EUXL, which produced an unexpected result.
On the 21st, G-EUXL is shown with its correct code as having been logged operating SHT7P !! The data also shows slightly different start and end times, and different start and end altitudes from the MyLog attach I sent previously for SHT7P.
It is also shown as having operated SHT6R the same day, which as I’ve said, did not appear in MyLog either.
I wouldn’t have the slightest idea whether this anomaly in logging is relative in any way to the ‘data being assigned .......’ bug being discussed here but perhaps it would be worth looking into at the same time. Alternatively, if it could be caused by some setting I have made, I would appreciate advice.
Syd
-
Mode-s Hex code, Squawk and Callsign will only be accepted when received fro the third time.
I guess my only question is whether THREE is the right threshold. Obviously the higher the threshold, the more "bad" data will be excluded, but the more "good" data may be lost. So, I did an experiment (yes, I know, I've got too much time on my hands!!!).
I have a recorded file of ModeS messages (using testmodes.exe), containing 232,263 messages. Within this there are 2,544 FlightID messages and 21,310 Squawk Code messages.
Out of the 53 HexCode/FlightID combinations received:
2 were received once only.
2 were received twice only.
49 others were received 3 or more times.
And out of 98 HexCode/Squawk combinations received:
3 were received once only.
2 were received only twice.
93 were receive 3 more more times.
So, albeit with a relatively small sample, with a requirement for 3 matches, that will represent a potential loss (if none were actually corrupted) of:
7.5% of FlightIDs.
5% of Squawk Codes.
Alternatively, with a requirement for 2 matches, that will represent a potential loss (if none were actually corrupted) of:
3.8% of FlightIDs.
3% of Squawk Codes.
I guess I'm just questioning whether THREE is the best threshold to use, or whether TWO would be better - any thoughts?
-
The way it works right now, witth the changes implemented, is:
1- Received mode-s code first time, do nothing;
2- Received mode-s code second time, do nothing;
3- Received mode-s code third time, add new aircraft;
After aircraft is added, flight number and squawnk will only be added when received for the third time after aircraft is added too MyFlights.
We can change these settings but we would prefer not have them as options which would confuse 95% of the users.
-
Please confirm you agre with our approach and the coding will start.
I don't agree with your approach. I prefer that you implement the proper solution using the parity check bits to clearly identify transmission errors. Everything else is just a workaround.
-
Please confirm you agre with our approach and the coding will start.
I don't agree with your approach. I prefer that you implement the proper solution using the parity check bits to clearly identify transmission errors. Everything else is just a workaround.
Hi Henning, have you read my comment on your last post? Using the parity bits may not solve the problem, if these checks are already implemented in a special decoding chip or in this case in the pre-processing chip. As far as I know today these kinds of parity checks are usually fully integrated in the hardware if a special decoding chip is used, or at least in the firmware if no special decoding chip is used (like this case).
For me the question is now: Are there already proper checks implemented in the firmware of the box, or are they not. If not, the firmware may be get (back) in focus, to add the checks there. If, yes go ahead in the already started way. On the other hand: From my point of view no serious programmer will leave out the opportunity to get rid off of corrupted data if he has a chance to do it at the front end of a signal/data processing chain. I would do it that way. But final confirmation can only be given by airnav, or the guy(s) who wrote the firmware to be more precisely.
@airnav: A lot of Software is using a the option to set to a professional mode, where additional configurations, that only a few users may need, can be altered. Maybe it is worse to consider to add this option to the upcoming new versions of RB Software. Because, I like the I idea to be able to access this parameter set-up in a direct way. It would give me the opportunity, to set them to a state were they are perfectly suiting my needs.
-
Henning,
As I stated earlier I am not tchnically minded but I have a question regarding your statement that parity checking will identify transmission errors.
Would it be the case that a parity check of a single transmission/packet could give a valid assumption of the wrong ID.
I ask as on the mode-s forum I see two or three times a week unknown ID's being made by different receivers and the answer given is usually misidentification of a bad packet. These are usually received once or twice by the individual receivers. Parity checks will have already been made I presume by the other box types and a further software processing appraoch as advocated would I presume further limit these misidentifications.
Examples
Probably corrupted packets rather than miscoded transponders.
4CA2D9/EI-DLM was airborne at the time you logged 4CA2DD
495145/CS-TJE was operating TAP663C while you had 495141 on your box
Another
> >
> > on the box in LHR right now is 400A2B which is G-JEDN (Dash 8), but based
> > on the flightlevel it must be a jet, probably an Embraer?
> >
> > Has anybody else seen this one before?
On Sep 21, 2012, at 14:59, "" <@...> wrote:
> Corrupted packets from TCX A320 G-KKAZ (400A0B) enroute IBZ-LBA.
>
> HTH
Alan
-
Hi Alan, I'll try to answer your question:
Parity checks are used to identify and in some cases to correct a corrupted data transmission. They are realised by adding additional bits to the actual data containing the information to be delivered. The bits are usually calculated from the payload data by using a simple up to a quite complex algorithm. The identification of a corrupted data package may work in the following way:
-After Receiving the message the receiver generates a own set of parity check bits from the received payload part by using the same algorithm the sender did use.
-After that the receiver compares its on parity bits with the received ones. If they match-up the data is considered valid and will be futher processed. If they do not match up it depends if error correction is implemented in the parity bits or not. If yes, the receiver can try to identify the bit that was altered and correct the data by toggle the bit number to its opposite state. If not the data will be dumped.
The problem is, that sometimes the messages is altered in a way, that the parity check bits are true although the data was corrupted. This will occur especially if the payload and the parity bits are altered within the same received data-telegram. For all these cases the additional stage as it is realised now, is a good way to filter out this kind of corruptions.
So, even if the parity bits are used for check-up whether a data-telegram is valid or not, it is still possible that strongly damaged data blocks with invalid information are able to come through. In this cases additional routines are needed to reduce this misidentifications. The routines added are such a suitable way.
How good the identification of damaged blocks works, depends on the complexness and resistance of the algorithm used to generate them.
Hope that helps a bit. (Too bad once I was much more firm in this topic, but by the years passing by a lot of data has gone with the wind...)
-
Helped me a great deal Ingo.
Alan
-
Hi Alan,
If you need some more technical background have a look at this website:
http://en.wikipedia.org/wiki/Cyclic_redundancy_check (http://en.wikipedia.org/wiki/Cyclic_redundancy_check)
Ingo, I am afraid that the RadarBox is not using a hardware parity check. On a competitor's product I haven't spotted any issues with miscodes messages.
But as you mentioned earlier, only AirNav can confirm this. We can just make assumptions.
-
Henning,
If that is the case how is the other box producing the errors in reply 23?. There are plenty of other I can direct you too. Go onto the mode-s forum and use the search option (if it works) and search on corrupt.
Alan
-
Corrupt can mean two different things:
1. The aircraft transponder can send out incorrect data - this cannot be handled by any receiver
2. During the transmission the original message gets corrupted (e.g. due to interference with other signals) - this can be identified using the parity check.
With the RadarBox I realised that sometimes information for one aircraft has been assigned to another aircraft maybe due to transmission errors. For example an aircraft that landed at an airport suddenly showed the same altitude that a high flying aircraft with a close by mode-s hex address had. And I never realised this happening with the other receiver.
But again: I assume that AirNav does not use the parity bits to check the integrity of the messages. But I cannot say if this is really the case.
Anyway, my comment was posted to give AirNav more information of how they can easily improve their product. Whether they wanna use this information or not is their choise.
At the moment I do not use my RadarBox at all as I am very disappointed because there are still no updates or bugfixes after years of waiting.
I also own a different receiver now and with that one I am quite happy at the moment. Although the UI does not look as nice as the RadarBox one. But a pretty interface does not help me if the data is not correct.
But if there will ever be an update of the RadarBox software I will definitely check it out. But that needs to be really good then after this long period of waiting for it. And not full of hacky workarounds.
Btw. are there any news about the exiting project which was announced to be released early September? It's nearly November now...
-
Henning,
The corrupt packages are as you describe is number 2, not sending out the wrong info in the first place. How is the other receiver getting it wrong if they are carrying out parity check which you detail sorts out corrupted packages.
Crap out/Crap in springs to mind with corrupted data. You cant on every occasion have 2 +2 making four, sometimes it may equal 3, 5, 10 etc giving the wrong answer
Alan
-
Alan,
We should stop guessing about what the RadarBox hardware and software does and what it does not. My initial post was information for the AirNav team. I am sure their developers are able to understand the information and are able to use it for their implementation.
I also don't want to analyze what causes incorrect data that is posted in other forums.
I simply wrote about what I realised using the two different receivers I own (and I cannot tell anything about other receivers).
-
I assume that AirNav does not use the parity bits to check the integrity of the messages.
Why make that assumption? This has all been discussed before, particularly the problem of not being able to parity check on some frame types without also receiving the ground station transmission requesting the aircraft data. See
http://www.airnavsystems.com/forum/index.php?topic=1641.msg28676#msg28676
and
http://www.airnavsystems.com/forum/index.php?topic=4916.msg49337#msg49337
My own theory is that this is a generic problem affecting any ModeS receiver that is not able to receive the transmissions from the ground station requesting the aircraft data in the first place, and so not able to check parity on all message types..
Just do a search for "parity" in this forum.