Gliffy diagrams backup tool

Delicious  is a useful tool. I use it in an unconventional way sometimes: I bookmark some resources with tags which serve as indicators for my backup tools. As an example, when I tag my Gliffy diagram pages with the my-diagrams tag, my backup tool grabs the corresponding diagrams and makes local copies on my server.

The backup tool I am referring to is readily available here.  I configure the backup tool to run periodically through cron jobs on my Linux server.

MindMeister map export

I use MindMeister almost daily. I wanted an easy way to backup my numerous mindmaps so I crafted a Python command-line tool for this purpose. It is available through this link and works under both Windows and Linux environments.  The tool performs logging in Windows registry or Linux /var/log/mm.log . For easier usage, the installer copies the command-line file to the  designated python script directory.

In summary, to use the tool (assuming one as a working Python installation,  3.0 <>=2.5 ):

Step 1) Register for an API key with MindMeister

Step 2) Install the tool through the command line:  easy_install jld

Step 3) Perform authorization: mm -s SECRET -k API_KEY auth

This step will open-up a browser window

Step 4) For regular backup, use a scheduler (such as Linux cron).

Suggestions are welcome!

Notes on Communication: Transfer Modes

Notes on Communication: some fundamentals

MindMeister: export to PNG bookmarklet

I use MindMeister mindmapping online application quite a lot. Very often, I need to embed an exported PNG image version of a mindmap in an other document.  This operation is cumbersome with the MindMeister currently so I decided to craft one solution available here.

Amazon Cloudfront CDN map

View Larger Map

Notes on Networking

I maintain several mindmaps on various subjects. This particular one contains my notes on networking and more specifically on the fundamental aspects of communications.

(Click on the bottom right-hand link to see full map)


Comments are appreciated.

MySQL load balancing

1) The traditional way of handling a two-tiered WEB architecture:

2) The software load-balancing approach:

3) The hardware load-balancing way:

With a versatile (i.e. scriptable) hardware load-balancer (e.g. F5's BIG-IP), one could scale the traditional architecture (see 1) without having to change the application. In this situation, one is trading software development effort against the cost of additional equipment.

Green Software: From Garbage Collection (GC) to Resource Recycling

I confess: I am a recycling freak.  When it comes to better our environment, I am all ears. 

I would like to offer a suggestion to the software community: in the green spirit, why couldn't we use the term Resource Recycling instead of referring to Garbage Collection?  That would help (albeit just a tad) setting the green reflex throughout the younger generations.

Python's __underscore__ operators

Cloud VPN service

With the advent of Microsoft's Azure service, I can see network operators have yet another good reason to ponder a Cloud VPN service. 

Example: Operator X offers Customer Y a service to connect all the customer's locations (with Ethernet Access and E-LAN for example) to Data-Centre(s) from Google/Amazon/Microsoft etc.

Why? The Internet is a great transport vehicule but still lacks QOS. As companies are migrating their applications to Web based ones, QOS might be a concern in some cases. 

Auto-properties for Subversion (Subclipse)

Too many times I had to dig the web to find how the Subclipse Eclipse plugin could be configured for auto-properties so I decided I would keep a record here.

1) Create a directory where the config file will be stored. On my development machine, I usually put it somewhere like worspace/configs/subclipse/config

2) Create a file named config in the directory in question:

<br />[miscellany]<br />enable-auto-props = yes<br />[auto-props]<br />* = svn:keywords=LastChangedDate LastChangeRevision LastChangedBy Id<br />*.jpg=svn:mime-type=image/jpeg<br/>
3) Finally, go to Windows->Preferences->Team->SVN and the Configuration Location to Use Directory and browse to the aforementioned directory created in step 1.

Data-Centres

Here are data-centres maps I track:

IBM

View Larger Map

Microsoft

View Larger Map

Google

View Larger Map

Add to Google Reader bookmarklet

Wow!  Google Chrome is out and of course I couldn't wait to try it out... that was until I discovered that automatic RSS/ATOM feed detection wasn't part of the deal.  I wrote a quick bookmarklet for addressing this situation: just drag-and-drop the following bookmarklet to your bookmarks (works on Chrome, Firefox and IE): 

The code is hosted on Google AppEngine so performance shouldn't be an issue. Have fun!

Cloud Computing Classification

The term Cloud Computing appeared to me too vague and thus I set out to investigate a bit further the question. As always, I resorted to my penchant for diagramming:

Design Pattern in PHP: Object Pool Pattern

Still working on my AOP framework for PHP, I stumble upon an interesting problem: when there are many files that require weaving, at some point memory usage was spiking very high (north of 256M) . For environments where memory is a scarce resource, I needed object recycling.

Here is what I came up with:

Design Pattern in PHP: Iterator Pattern

Whilst working on my AOP framework for PHP, I needed a versatile iterator. Here is what I came up with:

Javascript for PHP developers: Part I: Static Class

Here is how I would code a static class in Javascript by borrowing from my PHP experience:

Javascript

PHP

<br />// Function declaration<br />// Serves as "static class" declaration<br />// Will be attached to the 'window' object<br />// by default.<br />JLD = function() {<br /> // Static variable<br /> // Only accessible within 'JLD' <br /> var var1 = 666;<br />}<br /> <br />// Static Function declaration<br />// under the "static class" JLD<br />// Typeof JLD.fnc1 = "function"<br />JLD.fnc1 = function() {<br /><br /> // can't access var1 inside function JLD<br /> alert( "JLD.fnc1 called. var1= " + JLD.var1 );<br />}<br /><br />// 'fields' of JLD<br />JLD.var2 = 777; //typeof JLD.var2 = "number"<br />JLD.var3 = "some string"; //typeof JLD.var3 = "string"<br /><br />// Static Function declaration<br />// under the "class" JLD<br />JLD.fnc2 = function() {<br /> <br /> // can access var2 though<br /> alert( "JLD.fnc2 called. var2= " + JLD.var2 );<br />}<br /><br />

<br /> class JLD {<br /> <br /> function JLD() {<br /> var $var1 = 666;<br /> }<br /><br /> function fnc1() {<br /> // of course can't access $var1<br /> }<br /><br /> static $var2 = 777;<br /> static $var3 = "some string";<br /><br /> function fnc2() {<br /> // can access $var2 & $var3<br /> echo self::$var2;<br /> echo self::$var3;<br /> }<br />}<br />

Design Pattern in PHP: Factory pattern

The factory pattern is most interesting when Testing and Refactoring because one can substitute a class implementation for another without changing the core code.

In the example below, I have demonstrated a versatile factory method which sports object instance creation with a variable parameter list.

• Base::factory( $className /*optional parameters*/ ) creates an object of class $className with, optionally, a list of parameters passed to the constructor
• E.g. $object = Base::factory( 'TestClass', 'param1', 'param2' );
• Base::mapClass( $sourceClassName, $targetClassName ) adds a mapping entry so that the factory builds an object instance of $targetClassName instead of $sourceClassName when asked to build for $sourceClassName
• E.g. Base::mapClass( 'TestClass', 'TestDebugClass' );
  

Project
The project is located at GoogleCode.

Design Pattern in PHP: Template Method

This next design pattern should not be mistaken with the common template conception (see Web template) more commonly found in web-technologies.

The template method pattern is designed with a base class which contains an high-level recipe and relies on sub-classes to specify the ingredients. An example would be a template for publishing a table of objects: the template method base class would implement the publishing logic whilst the sub-class would extract the relevant data pieces to expose in the table.

Project
The project is located at GoogleCode.

Using Yahoo! Pipes with WEB-Services

I recently struggled a bit with Yahoo! Pipes when playing with this great tool trying to form a properly updating RSS feed from a WEB-Service source. The WEB-Service in question is available through one of Ohloh's API : Project.

The issue was related to the <guid> field required by (most?) RSS Aggregators (I use the excellent GoogleReader): fetching a data feed from a WEB-Service API usually means that no "proper" feed is in place and an RSS item must be created on-the-fly. This is no problem as such but when the RSS aggregator comes fetching updates, the <guid> field isn't updated dynamically by Yahoo! Pipes.

Part of the solution comes from using a mostly undocumented feature of Pipes: the y:id.value item tag.

After fetching the data from the data source, copy a field that can serve as good <guid> into the y:id.value field. In the example above, I used the updated_at field which, of course, isn't a <guid> per-se by not being global ; I'll find a better way next time around.


Finally, before connecting the "pipe output", insert the y:id.value in the pipes results as shown above.

For the interested, here is the pipe's address: Ohloh Project Summary as well as a link to all my pipes.

Have fun with Pipes... I know I do!

Design Pattern in PHP: Borg pattern

I thought about sharing some design patterns (expressed in PHP) I have documented. The first one is the borg. This design pattern serves as a good alternative option to the well-known singleton pattern.

The advantages of the borg are:

1. no need in client classes for special creation / access methods
2. easily sub-classed
3. more easily testable
   

Project
The project is located at GoogleCode.

Communication Types

There are 3 basic type of communication as illustrated in the following diagram:



Message
A message consists in a sequence of symbols, purposely chosen, from a lexicon. Note that the time-domain is not part of the information conveyed.

Open Sequence
An open sequence of messages carries information through the messages themselves but also through the order of the sequence i.e. the order in which the messages must be interpretated matters.

Timed Sequence
In a timed sequence, the time-domain adds information to the communication. Each symbol occupies, purposely, a temporal position in the ordered sequence. The receiving-end of the communication expects this timing to be respected in order to interpret the communication.

Network Functional Planes

Communication networks are usually modeled the following way:




User Plane
This plane supports the transport function i.e. the forwarding of traffic-units to end-point(s). The User-plane provides transport for the Control and Management planes.

Control Plane
This plane supports the control of the User-Plane with regards to the following aspects:

• Topology  ( virtual )
  
• Resources ( access etc.)

Of course the control-plane manages the virtual topology of networks unless one includes also "commands to humans to go dig-up streets and change patch-cords" i.e. humans are the only ones who control the fundamental physical topology (e.g. where cables are laid etc.).

Management Plane
This plane supports the management of the User and Control planes. The management functionality includes (see also FCAPS):

• Fault
• Configuration
• Accounting
• Provisioning
• Security

Network Protocol Stack Overview

The following diagram serves as reference.  One communication path is illustrated between the end-point E1 and E2. The communication traffic enters intermediate system A, is physically transmitted over to System B.

Interface "P"

This interface handles the "peer-to-peer" relationship between systems. In a communication system, the high-level semantic for this interface follows:

forward( traffic unit )

Where traffic unit (TU) represents the unit to forward to the end-point(s). On a packet network layer, the usual traffic unit is the packet (of course :-) whilst on an optical network layer the traffic unit will be an octet.

Interface "C"

This interface handles the "client-server" relationship between network protocol layers in a system. In a communication system, the high-level semantic for this interface follows:

forward( traffic unit, destination address, QOS )

The parameter destination address can either be explicit or implicit: this depends on the network layer in question. The parameter QOS (Quality Of Service) too can either be explicit or implicit depending the network layer technology.

Peer-To-Peer relationship

 The peer-to-peer relationship exists between separate (distinct) instantiations belonging to the same network protocol layer e.g. protocol sub-system N of system A and protocol sub-system N of system B. Physical proximity between sub-systems does not change the role of this interface.

Systems exchanging traffic through peer-to-peer relationships do so through a common address space. E.g. Internet Protocol. 

Client-Server relationship

The client-server relationship exists between a pair of network protocol layers. This relationship is characterize by a fundamental premise: the server-side of the relationship does not expose client-specific state i.e. the server must be designed to be "client-agnostic".

The client-side "adapts" the traffic it must send by mapping the desired destination address to the address space semantics of the server-side. The same logic is applied to the QOS parameter.

MindMeister online mind-mapping tool

I came across a truly wonderful tool yesterday night: MindMeister.

Example:



Have you found a better solution for online mindmapping? Let me know!

Eggs are to Pythons as Pears are to PHP.

With the advent of Google AppEngine, I am starting to look into Python programming because, let's face it, PHP support on AppEngine might be a long way off (feature requests).

It would appear that a good way of distributing Python Modules is through eggs which, from what I can understand, offer similar functionality as PEAR packages.

PythonEggs documentation can be found here.

MediaWiki extensions: 10's of them

For MediaWiki fans, I have made 10's of extensions readily available through either tarballs or PEAR.

For bug reports and features requests, please use the trac here. My user page on MediaWiki.org is available here.

Have fun!

PHP on Google AppEngine

For all the PHP fans out there, please vote for bringing PHP to Google's AppEngine. Use the official issue tracker here. Just star the issue and please do not insert inbox spamming comments (like "+1") in comments.

PROXY for PEAR channel

I recently submitted a bug report on PEAR regarding a long standing issue with the PEAR installer tool: there is no support for HTTP 302 (redirect). This condition effectively limits the hosting possibilities of PEAR channels e.g. a PEAR channel can not be accessible through a DNS cname aliases.

Problem

Suppose I have hosted a PEAR REST channel on Google Project @ http://mediawiki.googlecode.com/svn and now I would like to make this channel friendly through a nicer URL like " mediawiki.pear.jldupont.com ".
(note that the above is just an example: the channel on which I used the method described below in not yet publicly available)

Solution

The solution involves setting up a PROXY. This time around I have managed the feat with Apache using proxy_module 'mod_proxy.so' : in my vhost configuration, I have added:

<VirtualHost mediawiki.pear.jldupont.com:80>

ServerName mediawiki.pear.jldupont.com

DocumentRoot /var/www/vhosts/mediawiki.pear.jldupont.com

#PERFORMANCE tuning
#NOTE: not strictly required for the PROXY functionality
SetOutputFilter DEFLATE

ProxyRequests Off
ProxyPass / http://mediawiki.googlecode.com/svn/
ProxyPassReverse / http://mediawiki.googlecode.com/svn/

</VirtualHost>

1000's of Free Icons

I have amassed a reasonable collection of free icons. The collection is stored on Flickr and available here.

Zend PHP opcode disassembler

I have found a disassembler for Zend's PHP Opcode: Xcache (PHP opcode cacher) contains a PHP script which provides opcode disassembly services.

APC 3.0.17 and PHP 5.2.4

It would appear that APC 3.0.17 breaks down with PHP 5.2.4, that's at least my conclusion from upgrading my website's MediaWiki v1.11.0 installation.

PEAR Web Installer

Here is a quick tutorial on how to install PEAR's WEB Frontend installer.

1) Shell command: pear install PEAR_Frontend_WEB
2) Create file pear.php in the desired web directory with the following content:
<br /> require 'PEAR.php';<br /> require 'pearfrontendweb.php';<br />
I suggest putting this file in a separate folder with proper web server access permissions (i.e. htaccess, htpasswd etc.)
3) Adjust directory permissions for the PEAR system folders download and temp (on my LINUX system, these are located in /tmp/pear)
4) Adjust directory permissions for the PEAR classes folder (on my LINUX system, these are located in /usr/share/pear)

PEAR channels updated automatically

A marvelous new tool is available to the general public since yesterday: SproutBuilder. I decided to give it a try:






Above is a sprout automatically updated of all the PHP PEAR channels I know. The list is composed of entries I add to my Del.icio.us links with the tag pear-channel. Then this list is processed through a YAHOO! Pipe and finally embedded in the ''sprout''.

FeedBurner Awareness Data as RSS feed

I grew tired of logging on my FeedBurner account page to get my statistics data so I devised a Yahoo! Pipe to automate things. The Pipe can be used on any publicly accessible FeedBurner feed: here.

1and1 Linux VPS

It would appear that 1AND1 Linux VPS come now with Fedora Core 6 - 64bits. I really don't mind except when:
- PEAR isn't installed
- PECL neither (of course) ... and does not work even after fixing PEAR

Some good news though: yum is installed by default. Hooray!

Installing PEAR
Step 1: cd /tmp
Step 2: wget http://pear.php.net/go-pear
Step 3: php go-pear
Step 4: When prompted for the installation prefix, use /usr/share
Step 5: Try-out the pear command: pear

Installing PECL extensions
Getting PECL extensions to compile is a little bit more tricky...
Step 1: yum install gcc
Step 2: yum install php-devel

Even with this effort, I wasn't able to use the pecl command and had to resort to go through the pear command instead. See example next.

Installing PECL / APC
Step 1: pear install pecl/apc
Step 2: cp /usr/lib/php/modules/apc.so /usr/lib64/modules/php/apc.so
Step 3: vi /etc/php.d/apc.ini
Step 4: following step 3, edit apc.ini with extension=apc.so

Optimistic and Pessimistic Creation

Say one needs to create objects in one namespace in a controlled fashion (i.e. avoid duplicates, avoid loss); what strategy can one employ?

The Problem

The basic issue one has to deal with comes from the fact that possibly multiple processes are actively accessing a given namespace thereby collisions can occur: in namespace N, process 1 creates object O1 whilst process 2 does the same (i.e. creates object O1).


The left-hand side diagram illustrates the problem: the competing processes race to win the creation event during which time a meta-stable state exists i.e. no one process can be certain (to a reasonable degree) which will win the race.




Two main strategies
There are (at least) two main strategies to address the problem: an optimistic and a pessimistic one.


The optimistic strategy can be used in patterns where a process is considered reasonably alone in accessing a given namespace. Example: a user accessing is private file folder.

The pessimistic strategy must be employed in situations where multiple processes potentially access a given namespace. Example: an application's user creation flow.

Optimistic Strategy
The optimistic strategy consists in a simple procedure. The system:

1. verifies if object[X] exists
2. creates object[X] if it does not exist

The notation object[X] just means: an object with for identifier X.

Pessimistic Strategy
The pessimistic strategy is outfitted with a probabilistic equivocation reduction process. The system:

1. verifies if object X exists in the namespace
   
2. generate a reasonable unique identifier UID
   
3. creates a new identifier [X;UID]
4. verifies if an object with for identifier [X;UID] exists in the namespace
5. following #4, if no object exists, wait for Y units of time
6. verifies if any object with identifier prefix X exists
7. if the process' object with [X:UID] comes as first created (assuming multiple hits), then the process wins the creation race

An Application: Amazon S3 WEB Service

Say an application must be crafted with a system for registering users. Obviously, this process must be controlled: the system must be designed to deal with conditions such as:
Use Case: user X tries to register to the application with user-name UN1 at the same time as user Y goes for UN1.

One way to deal with the creation of unique user records on Amazon S3 I propose is:

1. Generate a long unique identifier (e.g. 32bytes)
2. Prepare a string identifier I/UID where I is the wanted identifier
3. PUT-OBJECT with for URI /I/UID
4. Wait for Y seconds (e.g. 3seconds)
5. LIST-OBJECTS with for prefix /I
6. If the object with for prefix /I/UID comes out to be the first created, then the process wins

The advantage of this algorithm stems from the fact that only 2 HTTP requests are necessary for each creation instance.

Enabling APC for PHP CLI scripts

Include the following lines in the php.ini script corresponding to the one used by PHP for running CLI scripts:

apc.enabled = On

apc.enable_cli = 1

Don't forget that the initialization file for the CLI operation of PHP is most probably different than the one used with Apache.

Using Google Alerts as Forum Notification method

One irritant on the WEB I found a solution for: have you ever posted questions or comments to a site that didn't include automatic email notification when the question/comment got feedback? It happens to me all the time.

Google Alerts to the rescue!

I now use GoogleAlerts to track my whereabouts on the WEB. The usage pattern is simple: just form the habit of always signing your post with a distinctive mark and set an alert on this mark. With my name, this is easy as it is not so common.

Greasemonky Quick Search

I have to sift through countless RSS feeds all day seeking more information based on keywords I find. I wanted to automate the process just a bit so I devised a GreaseMonkey Firefox user script.

Usage is simple: just select the text to search and press " alt c " which will automatically bring-up the result page on Google Search.

One can find the script here on UserScripts.org.

Embedding FreeMind mindmaps

FreeMind is a very handy (and free ;-) tool. I recently started playing around Google Mashup Editor and wanted to document in my way the API; I thus used FreeMind yet again.

I have embedded the mindmap in a Google Page available here.

Embedding the mindmap is easy:
<div id="mm" src="http://jeanlou.dupont.googlepages.com/GME.mm" width="800" height="800"></div>

and place the Javascript code at the bottom of the page:

<script src="http://jeanlou.dupont.googlepages.com/freemind.js" type="text/javascript"></script>

Syntax Highlighting

I have devised a straightforward way of performing syntax highlighting of Javascript code. Follow this link to the demonstration page: http://jeanlou.dupont.googlepages.com/syntaxhighlighting.

Javascript Class template

Here is a sample class template in Javascript:
<br />// constructor<br /> function obj(){<br /> // code here<br /> }<br /><br /> // properties<br /> obj.prototype.p = {<br /> var1: true,<br /> var2: false // no comma for last element<br /> }<br /> <br /> // methods<br /> obj.prototype.method = function() {<br /> // code here<br /> }<br />