From a valid RDF/XML for Twitter users to a dynamic SPARQL Endpoint

This weekend I upgraded the semantic profiling framework. Now it annotates for every Twitter user:

• Its profile as SIOC UserAccount
• The timeline as SIOC(Types) MicroBlog
• All the tweets as SIOC(Types) MicroBlogPost

It grabs the tweets from a user in Grabeeter if the user has registered there. If not they are being retrieved with the Twitter API.

The Twitter data extraction begins!

Today I started with the implementation of the extraction package. The package contains two models: one for the user’s profile and one for the user’s tweets. They then will be annotated in the “Annotator” module and converted into a list of simple triples in the “Triplifier” module. It is being implemented in PHP. There is a very good API: PHP Twiter with OAuth to help with this task. 

End of the easy learning curve…

Yesterday I was surprised by the ease which I could parse a twitter user profile and a tweet. But soon I ran into the complexities of the system. When I wanted to parse a not-supported xml format, I got no interesting results – I got lost in the documentation. The system is very easy to deploy and get results from the built-in rdf mapper. But developing an application (whether PHP, .NET or Java) requires very accurate architecture and deeper understanding of the system and its interfaces. Tough this shouldn’t be a problem, but for now the Virtuoso story ends.