Jun 2, 10:59 PM by Steve | Fixing Corrupted Garmin TCX Files
Posted in Cycling & Data
I guess I’ve been getting a bit of reputation as a magical fixer of corrupted TCX files. If this doesn’t mean anything to you, then don’t worry. This is all because the new firmware for the Garmin Edge 705 is not writing it’s data files correctly in some circumstances. Fortunately the nature of the TCX file format makes the bad files easy to fix if you understand XML. Unfortunately although it is possible to make the files readable again, there is usually some (or a lot) of data lost.
I can’t really explain XML here but the basic idea is that it consists of nodes enclosed in pairs of opening and closing tags. Every opening tag must have a matching closing tag. Every data sample in the file is a node called a trackpoint. For example:
<Trackpoint>
<Time>2009-05-31T14:57:58Z</Time>
<Position><LatitudeDegrees>42.290926</LatitudeDegrees><LongitudeDegrees>-71.389760</LongitudeDegrees></Position>
<AltitudeMeters>61.804</AltitudeMeters>
<DistanceMeters>92.244</DistanceMeters>
<HeartRateBpm><Value>94</Value></HeartRateBpm>
<Cadence>4</Cadence>
<SensorState>Present</SensorState>
<Extensions><TPX xmlns="http://www.garmin.com/xmlschemas/ActivityExtension/v2"><Watts>26</Watts></TPX></Extensions>
</Trackpoint>
So the basic building block of the TCX file is the Trackpoint, but there is a hierarchy to the file. One or more Trackpoint nodes are enclosed in a Track node and, one or more Track nodes make up a Lap node. A new track is created whenever the unit auto-pauses. Finally, at the top level, an Activity node contains one or more Lap nodes. There is only one Activity per file.
When the corruption happens, there is typically a trackpoint that has failed to write correctly and then the rest of the track is missing until the next track or lap starts. Because there is no matching closing trackpoint, track, and/or lap tags, the file is invalid and cannot be read by most programs. The “fix” is to remove the incomplete trackpoint and close out the track and/or lap. There is usually only one error like this but I’ve seen a few files that have more than one.
The first step to actually fix the file is to open the file in an editor. You can just open it in Notepad as it is a text file, but it will be difficult to find the bad nodes. I’ve found First Object XML Editor to work very well for our purposes, and it’s free to boot. This editor is nice because it can still parse the file and display the heirarchy even with the bad nodes, and as a bonus it highlights the bad nodes so they are really easy to find. With the file open, expand the nodes until you find one that is colored yellow, such as the Track node pictured here. This node is beginning of the problem as it does not have a closing tag, but it is not the end of the problem. Do not simply delete this node in the tree, it will not fix the corruption!
The real problem here is the lack of the closing tag, so scroll down the tree until you see a Trackpoint that is colored yellow, and the start of the next Track. If you double click on the last item before the next good Track it will highlight it in the text on the right.
Delete everything from after the last closed Trackpoint to before the start of the next Track, and replace with a closing Track tag as pictured below. Note that if it is the last Track in a Lap you may also need to close the Lap. Save the file and refresh and the tree should collapse and display nicely. There may be more errors so repeat the process.
So those are the basics for fixing TCX files. Some errors are a little harder to find because of their location but the principles are the same. If you have a file that you cannot fix feel free to send it to me, but please zip it first.
I’ve had corruption once so far. In my case the TCX file wouldn’t write. The writing history would briefly pop on the device screen and go away with no TCX file written, wouldn’t get past 4%. I couldn’t move the .dat file for the lap off the device either. After mounting and unmounting the 705 a 16K TCX file was finally written, but it was basically blank. The history on the device somehow remembers the average speed/power, etc. Sounds like I ran into a different corruption than what you describe to fix.
Now I’m going crazy why/how the distance and time is difference from what the 705 reports compared to the PT head unit.
— Weiland at Jun 3, 07:52 AMI am having a slightly different problem. I have lap .dat files that come up as a blank gray line in editpad. Before I delete them does this mean I have no data to even try and fix?
— Peter Stahl at Jun 30, 05:22 AMHi there magical fixer of files. I need help fixing this ride I did today. I tried to follow your directions, but I was unable to fix the problem. If you could help me, it would be much appreciated. Thanks.
— marn at Jul 3, 11:26 PMHi I seem to be having problems with my tcx file- and it seems you are a magical file fixer. If you could help that would be great- i did try a few things myself but… well it never worked..
— Jason at Jul 9, 04:41 AMCheers,
Jason
I seem to have such a problem, but can’t locate the bad node (expanding all the nodes would take ages!. any further ideas?
— Tom Phillips at Jul 16, 03:47 PMThanks so much! Was struggling to get my first Garmin file uploaded and Google brought me on your page. Didn’t understand much about the content, but downloading the XML editor and deleting these yellow lines saved the day (though, now I re-read your article, it shouldn’t have… well it worked anyway). Thanks mucho, and keep biking!!
— Olivier in Waterloo, Belgium (the real Waterloo) at Aug 1, 01:47 PMMy garmin saved the files in History/laps the laps folder contains 10 lap/ DAT files what to do? Help?
— Craig at Aug 9, 05:15 PMYou can use google for “xml validation service” and find some online service which validates xml. This way, you can easily find problems, as they usually show lines, where the problem happens
— PPershing at Sep 18, 11:10 PMHello, I tried to follow your instructions to fix a corrupt file, but I am having a hard time fixing the problem. Would you mind taking a look at my corrupt files (the ride was saved as 13 lap.dat files, but I think I have only found problems in 3 of the files. Thanks in advance.
— David at Dec 29, 02:17 AMScratch the prior comment, I was able to fix the file. Thanks for the post on this, very helpful.
— David at Dec 29, 08:12 AMI’ve got a tcx file that seems to be corrupted, but I’m incapable of fixing it as I’m technically illiterate. At your web site you kindly offered to have a look if the file is sent to you. If you are still willing to do that, please tell me where to send it. Thanks.
— Drew at Feb 11, 03:05 PMI am working on a really cool analysis website for cyclists, and we accept uploaded TCX files (GPX and KML too), as well as Garmin Communicator support. I am noticing a ton of corrupt files coming in from Garmin units, and I figured it was something to do with a connection. In any event, I am now automatically fixing those corrupt files when they are uploaded. Steve, since you are a software developer you can appreciate this: libxml2 comes with a command line utility called xmllint, which performs the fixing. Just wrapping up that functionality.
In any event, come check it out at http://ridewithgps.com
— Cullen King at Feb 19, 12:13 AM