Dear hivemind (especially jkpate!):

Is there anything resembling an open standard for corpus data, especially with interlinear glossing?

After years of frustration, I've finally reached my breaking point with the language documentation tools on the market. I think I can find a bit of money to get some new software made, but I'll still need to make most of the major structural decisions myself. To that end, I'd prefer not to reinvent the wheel. But, this certainly isn't my area of expertise, and I find myself (unsurprisingly) disoriented in a sea of proposals and data ontologies. I can read an XML schema well enough, but I don't have any sense of what people actually working with this stuff have and haven't been using successfully.

I suspect my needs aren't that specialized...quite the contrary; I'm surprised there's not already a clear standard here. Linguists and herding cats, I suppose. If anyone has had positive experiences with a data model, though, I'd love to hear about it. Especially important are:
* Comprehensive markup, including time alignment with audio files
* Compatibility with modern tools (xml beats the weird latex-like command tags that SIL uses, for example)
* Flexibility (These tools will be used for both linguistic analysis and more conventional translation projects, so it would be nice to have multiple tiers of analysis that can be used flexibly)
* Human readability (we'll probably use something like git for data versioning/collaboration, and at least early on most of the commits will be handled manually off of raw data)

I can get more specific as required. All help much appreciated!

The Nite XML toolkit looks absolutely beautiful. I'll dig into the nitty-gritty over the next few days, but the design overview sounds like exactly what I wanted...better than what I wanted, actually, because there's all sorts of stuff I would have never come up with on my own. There's even a query syntax!

Thanks for your help.

If you're going to use something like XML — and although I don't personally like XML very much, it is indeed a stable and mature format that is used widely — then it's all about the structure ... you can throw any idea about making any software (right now) straight out the window!

Making a graphical interface to work with your XML formats is going to be the easy part.

Indeed, at least relatively speaking, which is why I'm hoping somebody else has done the hard part for me already!

XML can be cumbersome because of how verbose it is, but that actually solves exactly the problem I'm dealing with here. There is no shortage of language documentation tools, but each one has a lot of baggage, ossified from the cognitive and methodological habits of its creators. I have no intention of designing tools that will work for all people at all times, but if I keep the UI relatively thin and build on flexible data models, others can more easily package their own (extensible and interchangeable) solutions as they see fit.

The reality is that most of the tools that currently exist were designed before most people had heard of the internet. These days, my typical informant is a subsistence goat herder with a smart phone.

Thanks for the links!