@adewale: yes this was the initial effort. Conceptually, this isn't rocket science, it is just putting the right pieces together. I've taken much of the ideas here and put them to work in the anyMeta sites we make at Mediamatic Lab. We have a federated network of social networking sites, including cross-site friending, creating works at sites that are not your home, and moving and combining profiles. Implemented using OpenID, OAuth, XRDS-Simple, Atom and XMPP/PubSub.
I understand the choice by Evan and the rest of the laconi.ca team to go with HTTP-only federation. It is easier to grasp and probably easier to implement for web-oriented developers. With that come scaling issues, most likely.
In the end though, I think that a microblogging service, or any type of service that deals with streams of social objects, is that it is essentially a message oriented application. A natural choice for the core of such applications is a messaging infrastructure, and XMPP comes into the picture.
Unfortunately, though, developing such systems, with their typical asynchronous nature, is not particularly easy, especially if you come from a web-oriented background. Also, I would like to state that there simply is no good code base to start from, yet. XMPP server implementations are typically designed for IM, and bots as XMPP clients just don't cut it.
A hybrid approach is very much possible, and that is what we've done at Mediamatic Lab. I enhanced my XMPP PubSub service Idavoll with an HTTP Interface to do just that. Stateless applications (like a web site implemented in PHP or Django in Google App Engine), can communicate via HTTP to get stuff done in the XMPP world and the other way around with callbacks. This particular interface is still a bit abstract and general purpose, and I am working on a slightly tighter integration. Also, it is a server-side component, and not really an XMPP face on the service, and I'd like to move it in that direction by finishing my XMPP server code in Twisted.
I explicitly didn't choose a BOSH like approach, because that would require too much XMPP knowledge on the non-XMPP side.
You're essentially blurring the distinction there between an XMPP interface onto a web service, and a web interface onto an XMPP service - I can't help feeling that the latter si the right way, and I get the impression that's the route you've chosen.
@dwd: true. And this is because I suspect the change of Jaiku from its current implementation to the GAE one to be a change from a message based architecture (although in a store-and-forward fashion, hence higher latency) to a web-based application that interacts with its long-running features through a dedicated HTTP API.
In our anyMeta system, the core thing is a semantic-web-like CMS that communicates with peers and could have messaging features. For most use-cases, it makes sense that it is an HTTP-based service with XMPP slapped on top of it.
On the other hand, I believe that starting from a messaging infrastructure is the best approach for a microblogging service. However, I don't think that one approach rules out the other, and there could be a large overlap in the infrastructure used.
A GAE approach would likely take in posts and comments through a regular web page and store stuff in a backend. It would then also send out notifications through an HTTP or some other light-weight, non-XMPP protocol (GEA is stateless) to long-running back-end services to do SMS, e-mail, XMPP chat message, XMPP PubSub, Pings and the mobile client. Some of these might instead be part of the GAE platform as an API. In reverse each of these other interfaces feed stuff back into the backend through an HTTP API that is part of the GEA code.
A true messaging based approach with have communicating services (much like Jaiku is now) with a storage facility that the HTTP interface runs atop of. The latency of posts on one interface being notified through other interfaces might be lower if you don't use store-and-forward.
You are arguing that these are inherently different. I think the only difference is in how the distribution of the notifications happens and when stuff gets stored.
I think there's a difference in approach, and that's the key. Aside from anything else, both XMPP and HTTP are merely interfaces onto the internal system, fo course, but it strikes me that if you approach the system as an XMPP service with a web front-end, you'd get something more likely to be scalable. But these needn't be the only interfaces, of course - as I said to you just now, this particular discussion would be much easier for me as an ad-hoc mailing list, and the email metaphor lends itself well to archival access, something that XMPP really isn't good at.
16 comments so far
Sorry about that, but then again, it might not be too late.
10 months, 1 week ago by ralphm
Was this the Twitter - Jaiku XMPP bridge?
10 months, 1 week ago by adewale
@ralphm: would u consider working on the FOSS version with some extra community devs
10 months, 1 week ago by BUGabundo
@BUGabundo: You're considering building something with the Jaiku Engine? O.o
10 months, 1 week ago by edythemighty
@adewale: yes this was the initial effort. Conceptually, this isn't rocket science, it is just putting the right pieces together. I've taken much of the ideas here and put them to work in the anyMeta sites we make at Mediamatic Lab. We have a federated network of social networking sites, including cross-site friending, creating works at sites that are not your home, and moving and combining profiles. Implemented using OpenID, OAuth, XRDS-Simple, Atom and XMPP/PubSub.
10 months, 1 week ago by ralphm
@BUGabundo: I would.
10 months, 1 week ago by ralphm
@ralphm: hummmmmmm, xmpp > http bridge or BOSH, that is the question
10 months, 1 week ago by cubicgarden
@ralphm i'm not. but seen ppl who wanted! even @evan/identica wanted some more info on this
10 months, 1 week ago by BUGabundo
This stuff is also similar to what @blaine cook has been working on since he left twitter.
10 months, 1 week ago by rabble
@cubicgarden: A good question.
I understand the choice by Evan and the rest of the laconi.ca team to go with HTTP-only federation. It is easier to grasp and probably easier to implement for web-oriented developers. With that come scaling issues, most likely.
In the end though, I think that a microblogging service, or any type of service that deals with streams of social objects, is that it is essentially a message oriented application. A natural choice for the core of such applications is a messaging infrastructure, and XMPP comes into the picture.
Unfortunately, though, developing such systems, with their typical asynchronous nature, is not particularly easy, especially if you come from a web-oriented background. Also, I would like to state that there simply is no good code base to start from, yet. XMPP server implementations are typically designed for IM, and bots as XMPP clients just don't cut it.
A hybrid approach is very much possible, and that is what we've done at Mediamatic Lab. I enhanced my XMPP PubSub service Idavoll with an HTTP Interface to do just that. Stateless applications (like a web site implemented in PHP or Django in Google App Engine), can communicate via HTTP to get stuff done in the XMPP world and the other way around with callbacks. This particular interface is still a bit abstract and general purpose, and I am working on a slightly tighter integration. Also, it is a server-side component, and not really an XMPP face on the service, and I'd like to move it in that direction by finishing my XMPP server code in Twisted.
I explicitly didn't choose a BOSH like approach, because that would require too much XMPP knowledge on the non-XMPP side.
10 months, 1 week ago by ralphm
You're essentially blurring the distinction there between an XMPP interface onto a web service, and a web interface onto an XMPP service - I can't help feeling that the latter si the right way, and I get the impression that's the route you've chosen.
10 months, 1 week ago by dwd
@dwd: true. And this is because I suspect the change of Jaiku from its current implementation to the GAE one to be a change from a message based architecture (although in a store-and-forward fashion, hence higher latency) to a web-based application that interacts with its long-running features through a dedicated HTTP API.
In our anyMeta system, the core thing is a semantic-web-like CMS that communicates with peers and could have messaging features. For most use-cases, it makes sense that it is an HTTP-based service with XMPP slapped on top of it.
On the other hand, I believe that starting from a messaging infrastructure is the best approach for a microblogging service. However, I don't think that one approach rules out the other, and there could be a large overlap in the infrastructure used.
A GAE approach would likely take in posts and comments through a regular web page and store stuff in a backend. It would then also send out notifications through an HTTP or some other light-weight, non-XMPP protocol (GEA is stateless) to long-running back-end services to do SMS, e-mail, XMPP chat message, XMPP PubSub, Pings and the mobile client. Some of these might instead be part of the GAE platform as an API. In reverse each of these other interfaces feed stuff back into the backend through an HTTP API that is part of the GEA code.
A true messaging based approach with have communicating services (much like Jaiku is now) with a storage facility that the HTTP interface runs atop of. The latency of posts on one interface being notified through other interfaces might be lower if you don't use store-and-forward.
You are arguing that these are inherently different. I think the only difference is in how the distribution of the notifications happens and when stuff gets stored.
10 months, 1 week ago by ralphm
I think there's a difference in approach, and that's the key. Aside from anything else, both XMPP and HTTP are merely interfaces onto the internal system, fo course, but it strikes me that if you approach the system as an XMPP service with a web front-end, you'd get something more likely to be scalable. But these needn't be the only interfaces, of course - as I said to you just now, this particular discussion would be much easier for me as an ad-hoc mailing list, and the email metaphor lends itself well to archival access, something that XMPP really isn't good at.
10 months, 1 week ago by dwd
@dwd: agreed.
10 months, 1 week ago by ralphm
@ralphm Have you seen this: http://openmicroblogging.org/2009/01/08/a-new-site-and-renewed-purpose/
10 months, 1 week ago by adewale
@adewale: Yeah, sure.
10 months, 1 week ago by ralphm