tag:blogger.com,1999:blog-32851970662054976952024-03-05T17:42:27.965-05:00Jean-Lou Dupont's WEBlogNotes on Cloud Computing, Telecommunications & SoftwareAnonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.comBlogger89125tag:blogger.com,1999:blog-3285197066205497695.post-82886234050298746252012-09-08T14:32:00.001-04:002012-09-14T05:52:11.413-04:00Notes on Hadoop on Amazon Elastic MapReduceI am on a project which requires Hadoop for crunching millions of documents. Being new to Hadoop, I was faced with a learning curve. Here are a couple of notes on the experience.<br />
<h2>
Hadoop Streaming</h2>
Working with streaming jobs on Hadoop is straightforward enough, except when it comes to controlling how Hadoop treats the input files. <br />
<br />
The input files I needed to process were large (100's MB). Hadoop insisted on chunking those to smaller size but unfortunately the default splitting wasn't compatible with my file type. I ended up fooling Hadoop in accepting bigger minimum split size:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">-D mapred.min.split.size=10737418240</span><br />
<br />
<h2>
Hadoop with Custom JAR</h2>
Although one has full control of Hadoop when using custom JAR jobs, I must say I had quite a rocky ride getting my stuff to work.<br />
<br />
First off, I needed a a way to package the dependent JAR librairies inside the main JAR. I was forced to write a custom ANT build file to do this.<br />
<br />
<pre class="php" name="code">
<target depends="compile" description="generate the distribution" name="emr">
<delete>
<fileset dir="${build.dir}" includes="*.jar">
</fileset></delete>
<jar basedir="${build.dir}" jarfile="${dist.dir}/mrlda-${version}.jar">
<zipfileset dir="./lib" excludes="hadoop*.*" includes="**.jar" prefix="lib">
</zipfileset></jar>
</target>
</pre>
<br />
<br />
The trick is to get the dependent jars in the directory<span style="font-family: Courier New, Courier, monospace;"> ./lib</span> for Hadoop EMR (I don't know about the standard Hadoop, sorry) to be able to include those in the Java classpath.<br />
<div>
<br /></div>
<div>
Second, I wanted to use Amazon S3 as both input and output for files. The trick there is to grab <span style="font-family: Courier New, Courier, monospace;">FileSystem</span> objects based on the URI scheme of the files.</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> Path opath=new Path(outputPath);</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> FileSystem ofs=opath.getFileSystem(conf);</span></div>
</div>
<div>
<br /></div>
<div>
Forget a single of those and the whole job crashes and you end up paying Amazon for nothing... $100's of dollars down the drain ironing up this one.</div>
<div>
</div>Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com4tag:blogger.com,1999:blog-3285197066205497695.post-81390949501093125972012-02-22T09:08:00.000-05:002012-02-22T09:08:02.142-05:00Lego Style Software DesignMy company recently got the chance to work on a very interesting project (which unfortunately I can't divulge at the moment). What made the project fun to work on, aside from the product level challenges, was the fact that we were given much more architectural flexibility than usual.<br />
<br />
Amongst the work emerged the following:<br />
<br />
<br />
<iframe allowfullscreen="true" frameborder="0" height="749" mozallowfullscreen="true" src="https://docs.google.com/presentation/embed?id=1H4ml0O5nqrh-2eRWHxLr4I2A8Zn6eZ1m2dBuO-m6Vsc&start=false&loop=false&delayms=3000" webkitallowfullscreen="true" width="100%"></iframe><br />
<br />Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com0tag:blogger.com,1999:blog-3285197066205497695.post-14318054494564127452012-02-05T19:44:00.001-05:002012-02-22T08:54:23.220-05:00Python Functional ToolsI got the chance to come by a pretty cool project recently: Python <a href="https://github.com/phzbox/Moka" target="_blank">Moka</a>. It consists of functional programming friendly implementations of standard Python dictionary and list classes. Then it struck me: wouldn't be nice to have Erlang-ish pattern matching functionality to Python?<br />
<br />
So I crafted a small Python package scratching an itch I have had for way too long: function dispatching based on pattern matching. For those interested, here are the relevant links:<br />
<br />
<br />
<ul>
<li>Project on <a href="https://github.com/jldupont/pyfnc" target="_blank">Github</a></li>
<li>Main page on <a href="http://www.systemical.com/doc/opensource/pyfnc" target="_blank">Systemical</a></li>
</ul>
<div>
<br /></div>
<div>
Feedback welcome :)</div>
<div>
<br /></div>Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com0tag:blogger.com,1999:blog-3285197066205497695.post-19274723042630986252012-02-02T22:11:00.001-05:002012-02-02T22:11:28.298-05:00EC2 architecture notesI've updated the home page of <a href="http://www.systemical.com/" target="_blank">Systemical</a>: you'll find a bunch of useful links to documents:<br />
<br />
<br />
<ul>
<li><a href="http://www.systemical.com/notes/ec2-arch" target="_blank">An EC2 architecture</a></li>
<li><a href="http://www.systemical.com/doc/system/election-protocol" target="_blank">Distributed Leader Election Protocol</a></li>
<li><a href="http://www.systemical.com/notes/data-flow-ec2" target="_blank">Data-Flow based EC2 architecture</a></li>
</ul>
<div>
Enjoy :)</div>Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com0tag:blogger.com,1999:blog-3285197066205497695.post-53953226702034996232012-01-20T16:01:00.000-05:002012-01-20T16:01:00.105-05:00Amazon AWS tools - jldawsToday I am open-sourcing yet another project. It consists in a collection of Linux scripts related to Amazon Web Services (AWS).<br />
<br />
The project's home page can be found <a href="http://www.systemical.com/doc/opensource/jldaws">here</a> whilst the code repository is <a href="https://github.com/jldupont/jldaws" target="_blank">there</a>.Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com0tag:blogger.com,1999:blog-3285197066205497695.post-72579307029053432162011-12-08T14:27:00.001-05:002011-12-08T14:53:50.759-05:00My life in the CloudI am pretty much 100% in the Cloud now. Here's how I do it.<br />
<br />
<span style="font-size: large;">Services</span><br />
<br />
<ul>
<li>Email: Google GMAIL (with "Tasks")</li>
<li>Calendar: Google Calendar</li>
<li>Documents: Google Docs (with a sprinkle of DropBox, going away soon enough)</li>
<li>Music: combination of <a href="http://www.rdio.com/#/people/dupontjl/" target="_blank">Rdio</a> and <a href="http://grooveshark.com/" target="_blank">Grooveshark</a></li>
<li>Movies: Netflix (on my Android devices, Apple TV and MacBook Pro)</li>
<li>Code: <a href="https://github.com/jldupont" target="_blank">Github</a></li>
<li>Project Issue Tracker: <a href="http://lighthouseapp.com/" target="_blank">Lighthouse</a></li>
</ul>
<div>
What I gain:</div>
<div>
<ul>
<li>Mobility</li>
<li>Availability</li>
<li>Persistence</li>
<li>Version Control</li>
</ul>
<div>
I don't need to worry about loosing / forgetting my laptop for work: I just grab a spare one at work. Hence, business continuity is greatly improved over the traditional Microsoft "stack".</div>
</div>
<div>
<br /></div>
<div>
So, on my Linux/OSX PCs, I only need the following installed software:</div>
<div>
<ul>
<li>Google Chrome (the auto-sync feature for bookmarks, extensions etc. is way cool :)</li>
<li>Eclipse (with PyDev, Google Plugin, Egit)</li>
<li>git</li>
</ul>
<div>
When it comes time to setting-up a complete working environment, I can turnaround one in under <b>1/2 hour</b>! </div>
</div>
<div>
<br /></div>
<div>
<span style="font-size: large;">Security</span></div>
<div>
The first reaction I get when I expose my workflow: is this Cloud stuff secure? To which I respond with a combination of the following:</div>
<div>
<ul>
<li>I would rather trust Google and their army of sysadmins than me when it comes to external security threats</li>
<li>I never write down passwords nor credit card numbers in documents. Period.</li>
<li>Doing business comes down to trust - being part of society requires trust</li>
<li>My data is more secure located in the Cloud than on my laptop / desktop : those are too easy to lift</li>
</ul>
<div>
<span style="font-size: large;">Availability</span></div>
</div>
<div>
Another common reaction is about availability. I don't know about you guys, but Internet Access is pretty available nowadays and certainly Google is. For those of you worried about this dimension, consider:</div>
<div>
<ul>
<li>I have access through my Android mobile devices (carrier #1)</li>
<li>I have access through my 3G dongle (carrier #2)</li>
<li>I have access through my Cable Modem (carrier #3)</li>
</ul>
<div>
Yes I subscribe to 3 different carriers - I don't remember the last time I lost connectivity and was out of commission, business wise.</div>
</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com0tag:blogger.com,1999:blog-3285197066205497695.post-50442663167532548202011-10-29T21:59:00.001-04:002012-02-22T09:07:11.749-05:00Software based Message Switch<br />
<br />
<iframe allowfullscreen="true" frameborder="0" height="750" src="https://docs.google.com/presentation/embed?id=1_2VEkOGLK7z8nfUDLKSTZXbnvgLWaL7ffMDCz01MVLs&start=false&loop=false&delayms=3000" webkitallowfullscreen="true" width="100%"></iframe>Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com0tag:blogger.com,1999:blog-3285197066205497695.post-52231228550567299152011-08-14T09:03:00.002-04:002011-08-14T09:03:31.484-04:00Media-Keys over Web-SocketMy newest open-source project is a Media-Keys over Web-Socket application. The project's details can be found <a href="https://sites.google.com/a/systemical.com/v1/doc/opensource/mediakeysserver">here</a>.<br />
<br />
<b>Application</b><br />
The main use-case I wanted to address with this component is the ability to control music playback of web-sites such as <a href="http://grooveshark.com/">GrooveShark</a> and <a href="http://www.rdio.com/">Rdio</a> through the keyboard media-keys.<br />
<br />
<b>Usage</b><br />
A Google Chrome extension will follow shortly.<br />
<br />Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com0tag:blogger.com,1999:blog-3285197066205497695.post-31829618975449935672011-05-18T13:17:00.000-04:002011-05-18T13:17:30.386-04:00Using Google Sites as Content Management SystemI recently had to build a new web site from ground-up. One thing I didn't want to do is go off installing Wordpress / Drupal etc. on a virtual machine. As much as I love technology, I can't be bothered anymore with administrative tasks such as backups.<br />
I decided to give Google Sites another go but I quickly was confronted with the limitations of this platform. That's when I decided to couple to great Google products: Sites and AppEngine.<br />
<br />
View the <a href="http://www.systemical.com/doc/cms">presentation</a>Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com0tag:blogger.com,1999:blog-3285197066205497695.post-62221451670727151482011-04-15T19:33:00.000-04:002011-04-15T19:33:56.920-04:00A Service Registration Web-Service APIIt's been a while since I have last blogged... been very busy with my company <a href="http://www.data-tester.com/">Data-Tester Inc.</a><br />
<br />
I have the pleasure to announce the immediate availability of another Web-Service hosted on <a href="http://www.systemical.com/">Systemical</a> : <a href="http://www.systemical.com/web-services/ipreg">ipreg</a>. This service allows, for example, mobile devices (such as Android) to "discover" available LAN services). <br />
<b>Why is this useful at all you might ask?</b> For the curious: it boils down to limitations related to multicast support in mobile devices. For more details, read the details <a href="http://www.systemical.com/web-services/ipreg">here</a>.Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com0tag:blogger.com,1999:blog-3285197066205497695.post-5658295356220286792010-10-11T20:52:00.000-04:002010-10-11T20:52:37.117-04:00iTunes playcount updater scriptI am pretty busy these days with my new job ( I am heading a start-up in the telecommunications market ) but I took a couple of hours to craft a couple of scripts.<br />
<br />
The first script (<i>lastfm_gettracks</i>) downloads all of a user's song track information from <a href="http://last.fm/">Last.fm</a>. It can be found on Pypi <a href="http://pypi.python.org/pypi/jld_scripts">here</a>. More information can be perused <a href="http://www.systemical.com/doc/opensource/jld_scripts">here</a>.<br />
<br />
The second script takes as input the file generated by the former script and updates the iTunes database on an OSX machine (sorry, I don't do windozes). Documentation for this script can be found <a href="http://www.systemical.com/doc/opensource/itunes_update_playcount">here</a> whilst package details can be found on Pypi <a href="http://pypi.python.org/pypi/jld_osx">here</a>.Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com0tag:blogger.com,1999:blog-3285197066205497695.post-16265196527464179302010-09-09T15:34:00.001-04:002010-09-09T15:34:40.516-04:00Logo contestHelp me choose a logo for my new company. The logo contest on <a href="http://99designs.com/">99designs</a> has ended and it is now time to vote!<br />
<div style="text-align: center;"><a href="http://99designs.com/logo-design/vote-b48bii">VOTE HERE</a></div><div style="text-align: left;">Thanks!</div><div style="text-align: left;"><br />
</div>Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com0tag:blogger.com,1999:blog-3285197066205497695.post-26022615307464958052010-09-05T09:08:00.000-04:002010-09-05T09:08:49.623-04:00Chrome Framed extensionI have just published my latest Google Chrome extension: <a href="http://www.systemical.com/doc/opensource/chrome-framed">Chrome-Framed</a>. The extension allows opening a tab with 2 columns 'frame'. This functionality enables a user 2 pages side-by-side in the same chrome window.<br />
<br />
Enjoy!Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com0tag:blogger.com,1999:blog-3285197066205497695.post-82332491104886854572010-08-17T20:21:00.001-04:002010-08-17T20:52:53.098-04:00Musicbrainz and Last.fm proxies updatedJust a quick note to let users of <b><a href="http://www.systemical.com/doc/opensource/musicbrainz-proxy-dbus">musicbrainz-proxy-dbus</a></b> & <b><a href="http://www.systemical.com/doc/opensource/lastfm-proxy-dbus">lastfm-proxy-dbus</a></b> applications that new versions are available on my <a href="https://launchpad.net/~jldupont/+archive/jldupont">PPA</a>. Notable changes include:<br />
<br />
- Tray icon support: the main application window can now be closed and a tray icon can control them<br />
- Help button linking to a web accessible pageAnonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com0tag:blogger.com,1999:blog-3285197066205497695.post-60095079453119205432010-06-20T21:33:00.001-04:002010-06-20T21:33:39.503-04:00Web Services available on Systemical.comI've decided to open access to some web services I built. These services are hosted on <a href="http://code.google.com/appengine/">Google AppEngine</a> and the documentation can be found <a href="http://www.systemical.com/web-services">here</a>.<br />
<ul><li>Google Moderator: an RSS feed for a "series"</li>
<li>Stackoverflow: an RSS feed for a user's statistics on <a href="http://stackoverflow.com/">http://stackoverflow.com/</a></li>
<li>Github: an RSS feed for a user's statistics on <a href="http://github.com/">http://github.com/</a></li>
</ul>I will be opening access to more services so stay tuned!Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com0tag:blogger.com,1999:blog-3285197066205497695.post-85599665592703565132010-06-17T09:00:00.000-04:002010-06-17T09:00:33.354-04:00Last.fm Proxy with DBus interfaceI just realized I haven't made a public announcement regarding an application that constitutes a critical piece of my <a href="http://jldupont.blogspot.com/2010/06/rhythmbox-synclastfm-plugin.html">Rhythmbox arsenal</a>: " lastfm-proxy-dbus ".<br />
<br />
This Linux Gnome application serves as "proxy" to a Last.fm user's scrobbling history. The data collected through this proxy serves to populate the "playcount" field of Rhythmbox's database.<br />
<br />
More information can be found <a href="http://www.systemical.com/doc/opensource/lastfm-proxy-dbus">here</a>.Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com0tag:blogger.com,1999:blog-3285197066205497695.post-7612436210210353892010-06-15T15:13:00.003-04:002010-06-15T15:48:38.299-04:00Rhythmbox Synclastfm pluginToday I announce the availability of a brand new version (2.02) of my Rhythmbox plugin "Synclastfm" (see <a href="http://jldupont.blogspot.com/2010/01/rhythmbox-plugin-syncing-with-lastfm.html">previous post</a>).<br />
<br />
This version packs the much anticipated "batch update" feature i.e. the plugin will walk the user's history on Last.fm in order to update Rhythmbox tracks 'playcount' field.<br />
<br />
More information can be found <a href="http://www.systemical.com/doc/opensource/rb-synclastfm">here</a>. Have fun and don't forget to provide feedback!Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com12tag:blogger.com,1999:blog-3285197066205497695.post-33458236449825650152010-05-30T23:28:00.002-04:002010-06-10T08:44:02.356-04:00Musicbrainz Proxy service over DBusI have just finished another building block on my quest to taming my music library: a Linux based application serving as 'proxy' (with caching of course) to Musicbrainz webservice. The service is exposed through DBus.<br />
<br />
<b>Why?</b><br />
Correlating music tracks across disparate applications / services can be daunting. For example, suppose one <i>scrobbles</i> his/her tracks to <a href="http://last.fm/">Last.fm</a>. Furthermore, let's say one wishes to keep his/her favorite Music Player in <i>sync</i> with his/her Last.fm profile. This goal appears, on the surface, to be an easy one but as it turns out, support for finding & cataloging music tracks coherently across various applications / services remains elusive at the moment.<br />
<br />
Also worth noting: access to Musicbrainz's webservice is monitored. Webservice API access is limited to 1call/sec: if many applications make uncoordinated access to the service, the user's machine might get blacklisted. As I intend to make the <i>track mbid</i> a central piece of information for correlating my music tracks, I needed to address this issue.<br />
<br />
<b>Solution</b><br />
With this application, it will be easier to integrate the various other building blocks of my music management arsenal. See the other project <a href="http://jldupont.blogspot.com/2010/01/rhythmbox-plugin-syncing-with-lastfm.html">SyncLastfm</a> that will benefit from this addition.<br />
<br />
<b>Project</b><br />
The project's home page can be found <a href="http://github.com/jldupont/musicbrainz-proxy-dbus">here</a>. The associated <a href="http://ohloh.net/">Ohloh</a> page can be found <a href="https://www.ohloh.net/p/musicbrainz-proxy-dbus">there</a>.<br />
<br />
<b>Update:</b> the application will be installed under <i>Applications / Other.</i><br />
<b>Update2:</b> more information is accessible <a href="http://www.systemical.com/doc/opensource/musicbrainz-proxy-dbus">here</a>.Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com0tag:blogger.com,1999:blog-3285197066205497695.post-25476338163589673032010-05-10T13:26:00.001-04:002010-05-10T13:26:18.018-04:00Some Google GadgetsI am in the process of building up my <a href="http://www.systemical.com/">company website</a> based on Google Apps, I need some Google Gadgets to embed but as I am unsure about the majority of the gadgets listed in the <a href="http://www.google.com/ig/directory">directory</a>, I have created some of my own.<br />
<ul><li>Skype Account Information</li>
<li>Ohloh Profile Flair</li>
<li>Stackoverflow Profile Flair</li>
<li>Twitter Profile Flair with latest tweets</li>
</ul><div>Links to the relevant gadgets are provided on my <a href="http://jldupont.blogspot.com/2009/09/mindmap-of-my-projects.html">projects page</a>. I would rather not duplicate them here as I am most likely to forget to update them whereas on the project page they'll get my attention.</div><div><br />
</div><div><br />
</div><br />
<div class="zemanta-pixie" style="height: 15px; margin-top: 10px;"><a class="zemanta-pixie-a" href="http://reblog.zemanta.com/zemified/958ac04f-d3b7-4971-8fd8-41c76a776deb/" title="Reblog this post [with Zemanta]"><img alt="Reblog this post [with Zemanta]" class="zemanta-pixie-img" src="http://img.zemanta.com/reblog_e.png?x-id=958ac04f-d3b7-4971-8fd8-41c76a776deb" style="border: none; float: right;" /></a><span class="zem-script more-related pretty-attribution"><script defer="defer" src="http://static.zemanta.com/readside/loader.js" type="text/javascript">
</script></span></div>Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com0tag:blogger.com,1999:blog-3285197066205497695.post-18174213264476323952010-02-08T09:20:00.005-05:002010-02-08T09:26:21.082-05:00Last.fm Web Service desktop interface through DBusNot too long ago, I started my <i>quest to Music Library Faiyland</i> (my daughter would love this quote). On this journey, I have completed a couple of intermediate steps ( <a href="http://jldupont.blogspot.com/2010/01/rhythmbox-plugin-syncing-with-lastfm.html">Rhythmbox plugin - Syncing with Last.fm</a> and <a href="http://jldupont.blogspot.com/2010/01/rhythmbox-plugin-for-lastfm-desktop.html">Rhythmbox plugin for Last.fm desktop client</a> ) and now I am happy to announce another milestone: a DBus accessible interface to <a href="http://last.fm/">Last.fm</a>'s Web Service API.<br />
<br />
<span class="Apple-style-span" style="font-size: x-large;">API</span><br />
The current DBus interface is rudimentary but can be easily extended ( I have abstracted Last.fm's API to a manageable degree ). I will certainly be expanding the <i>methods</i> in the near future based on my needs and "customer feedback" ;-) Below is a non-exhaustive list of the API:<br />
<br />
<ul><li>Account related:</li>
<ul><li>get/set Username</li>
<li>getAuthUrl : retrieve a URL for authenticating the service against a user</li>
<li>setApiKey , setSecretKey</li>
</ul><li>Track related</li>
<ul><li>track.addTags</li>
<li>track.getTags</li>
<li>track.removeTag</li>
</ul></ul><br />
I have only implemented these <i>methods</i> because they get me what I need at the moment: the possibility to "rate" my music tracks through a <i>hack</i> using the tags to hold the rating for a track. E.g. a 5 stars track would get "R5" as tag.<br />
<br />
<span class="Apple-style-span"><span class="Apple-style-span" style="font-size: x-large;">Message Based Design Technique</span></span><br />
In this project, I have further experimented with a design technique based on "Message Passing". I have made <a href="http://jldupont.blogspot.com/2010/01/notes-on-message-passing-design.html">some notes on the subject here</a>.<br />
<br />
<span class="Apple-style-span"><span class="Apple-style-span" style="font-size: x-large;">Project</span></span><br />
The home page of this project can be found on <a href="http://github.com/jldupont/dbus_lastfm">my Github repository</a>. A Debian package can be found on <a href="https://launchpad.net/~jldupont/+archive/jldupont">my Launchpad PPA</a>.Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com0tag:blogger.com,1999:blog-3285197066205497695.post-12980021025804428852010-01-29T17:36:00.001-05:002010-01-29T17:39:29.890-05:00Notes on a Message Passing designI am an avid supporter of "message based" systems. I took the time to perform a quick "brain dump" of some of my thoughts on the subject.<br />
<br />
<iframe src="http://docs.google.com/present/embed?id=dgstxrxv_453cgxxmjc2&size=m" frameborder="0" width="555" height="451"></iframe><br />
<br />
Follows a simple Python based implementation.<br />
<br />
<script src="http://gist.github.com/288887.js"></script><br />
<br />
I will very likely make other contributions to my blog along the same lines.Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com0tag:blogger.com,1999:blog-3285197066205497695.post-91312771531539466252010-01-21T20:04:00.017-05:002010-01-21T20:30:11.090-05:00Message Bus for Gnome Object based Python projectsWhilst developing my latest software project (<a href="http://jldupont.blogspot.com/2010/01/rhythmbox-plugin-syncing-with-lastfm.html">Rhythmbox plugin Last.fm sync</a>), I have crafted a "Message Bus" based on Gnome GObject. Since I didn't have any experience at all with Gnome GObject previously, it took me sometimes to work out the details of the architecture. For an example usage, please refer to my <a href="http://github.com/jldupont/rb_synclastfm">Git Repository</a>.<br />
<br />
<pre class="python" name="code">"""
GObject based "Message Bus"
@author: Jean-Lou Dupont
"""
import gobject #@UnresolvedImport
class Signals(gobject.GObject):
"""
List of the application level signals
"""
__gsignals__ = {
## NOTE: customization required here
## =================================
## Announces changes in the user's Last.fm properties
"lastfm_username_changed": (gobject.SIGNAL_RUN_FIRST, gobject.TYPE_NONE, (gobject.TYPE_STRING,))
,'lastfm_password_changed': (gobject.SIGNAL_RUN_FIRST, gobject.TYPE_NONE, (gobject.TYPE_STRING,))
## Used to signal a change in the currently playing track
,"playing_song_changed": (gobject.SIGNAL_RUN_FIRST, gobject.TYPE_NONE, (gobject.TYPE_OBJECT,))
## Used to report a failure when accessing Last.fm web service
,'lastfm_request_failed': (gobject.SIGNAL_RUN_FIRST, gobject.TYPE_NONE, ())
## Used for distributing the results of the query against the Last.fm web service
,"user_track_info": (gobject.SIGNAL_RUN_FIRST, gobject.TYPE_NONE, (gobject.TYPE_OBJECT,))
## Used to pass around the "shell" global object
,"rb_shell": (gobject.SIGNAL_RUN_FIRST, gobject.TYPE_NONE, (gobject.TYPE_OBJECT,))
}
def __init__(self):
gobject.GObject.__init__(self)
class Bus(object):
"""
Message Bus
Borg Pattern
"""
_signals=Signals()
@classmethod
def emit(cls, name, *pa, **kwa):
cls._signals.emit(name, *pa, **kwa)
@classmethod
def add_emission_hook(cls, name, callback):
gobject.add_emission_hook(cls._signals, name, callback)
mbus=Bus()
## =============================================================== Tests
if __name__=="__main__":
def callback(signal, data):
print "callback: signal=%s, data=%s" % (signal, data)
Bus.add_emission_hook("lastfm_username_changed", callback)
Bus.emit("lastfm_username_changed", "jldupont")
</pre><br />
<span style="font-size: x-large;">Relevant Stackoverflow posts</span><br />
Here are two Stackoverflow posts in relation with my exercise: <a href="http://stackoverflow.com/questions/2074372/connecting-signals-in-gtk-anonymous-way">question 1</a>, <a href="http://stackoverflow.com/questions/2088451/gobject-addemissionhook-usage">question 2</a>.Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com0tag:blogger.com,1999:blog-3285197066205497695.post-7511961834653103572010-01-21T19:13:00.001-05:002010-06-15T15:15:47.523-04:00Rhythmbox plugin - Syncing with Last.fmIn my quest to manage my music collection seamlessly between media players, I've decided to use <a href="http://www.last.fm/">Last.Fm</a> as my main database. Last.Fm provides ways to store:<br />
<br />
<ul><li>History of played tracks</li>
<li>Playcount associated with played tracks</li>
<li>Tags associated with tracks</li>
<li>Playlists</li>
<li>etc.</li>
</ul><div>Through their API (see <a href="https://www.mindmeister.com/maps/show/39035072">mindmap</a>), it is possible to have access to all this information (and much more).</div><div><br />
</div><div><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: x-large;">Rhythmbox </span></span></div><div>My music collection is currently organized through Rhythmbox - it wasn't my first choice at the time I've moved from my Windows desktop to my Linux Ubuntu one but it has proven very useful. RB can be extended with Python scripts quite painlessly.</div><div><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: x-large;">Last.Fm Syncing</span></span></div><div>The plugin I have developed this time supports the following features:</div><div><ul><li>When a track is played, it downloads the corresponding information from the user's Last.Fm account and updates the following fields of the Rhythmbox database:</li>
<ul><li>Playcount: if the "last.fm playcount" associated with the track is greater than 0, the local playcount of RB is updated with it</li>
<li>Rating: if the "last.fm love" field is set for the track, the local "rating" field of the track is set to 5.0 (5 stars)</li>
</ul></ul><div>With this functionality, it is easier to update a local Rhythmbox media player database with information stored on a user's account on Last.fm. The standard Last.Fm plugin of Rhythmbox must be enabled and configured correctly.</div><div><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: x-large;">Installation</span></span></div><div>The plugin can be installed by following the procedure found at the <a href="http://github.com/jldupont/rb_synclastfm">project's home page</a>. A direct link to the plugin Debian/Ubuntu repository is <a href="https://launchpad.net/~jldupont/+archive/jldupont">provided here</a>.</div></div><div><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: x-large;">Future</span></span></div><div>The project is far from being completed - I have many other features in the pipeline. Stay tuned!</div><div><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: x-large;">Related project</span></span></div><div>There is a related project to this one: <a href="http://jldupont.blogspot.com/2010/01/rhythmbox-plugin-for-lastfm-desktop.html">Rhythmbox - Last.Fm Desktop Client plugin</a>.</div><div><br />
</div><div>Feedback is very welcome!</div><div><br />
<b><u>UPDATE:</u></b> see <a href="http://jldupont.blogspot.com/2010/06/rhythmbox-synclastfm-plugin.html">here</a> for details on the new release.</div><div><br />
</div>Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com15tag:blogger.com,1999:blog-3285197066205497695.post-21114388149082396742010-01-11T16:50:00.018-05:002010-01-11T20:29:00.746-05:00Rhythmbox plugin for Last.fm desktop clientI've recently lost my music database to the hands of Banshee (actually, the Sqlite3 database file is still somewhat salvageable). I have longed for an application that would make my life easy in managing my music collection (which is pretty extensive) and do this day haven't found one. The recent event pushed me to the edge: I am embarking on a quest to <i>scratch my itch on this matter.</i><br /><div><span class="Apple-style-span" style="font-size:large;"><b>First step</b></span></div><div>The first step consisted in getting acquainted with <a href="http://projects.gnome.org/rhythmbox/">Rhythmbox</a>, my new music media player. Why Rhythmbox? because it supports a Python based extension system (compared to Banshee's C# based on) and I happen to really enjoy writing software using Python.</div><div>To this effect, I have crafted a plugin that sends the current playing state (START/PAUSED/RESUMED) of Rhythmbox to <a href="http://www.last.fm/download">Last.fm's Desktop Client</a>.</div><div><span class="Apple-style-span" style="font-size:large;"><b>Plugin</b></span></div><div>The project is located on <a href="http://github.com/jldupont/rb_lastfmdk">Github</a>. </div><div><b><span class="Apple-style-span" style="font-size: large;">Project</span></b></div><div>I intend to make several other contributions focused on media players. My goal is to be able to synchronize my music collection seamlessly between Rhythmbox, Last.fm and iTunes (with possibly also <a href="http://musicbrainz.org/">Musicbrainz</a>). Stay tuned!</div>Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com0tag:blogger.com,1999:blog-3285197066205497695.post-29777647601266549422010-01-06T20:35:00.002-05:002010-01-06T20:36:37.244-05:00Google Wave InvitesI've got some Google Wave invites left... first come, first served!<div><br /></div>Anonymoushttp://www.blogger.com/profile/00409924945516479600noreply@blogger.com0