Build your own Search Engine

August 31st, 2008

Concord Dawn – Aces High (Featuring State of Mind)

Problem: You need to provide enterprise search for your users, but you have a budget of $0. The $tightwads at work won’t cough up for the best practice solution: Google appliances. You need something that will scale and perform, yet it won’t have all the advanced features or integration capabilities of a GSA.

OpenWebSpider does the standard crawling and writes its results to a mysql database. Sphinxsearch connects to any other db’s and indexes those, as well as indexes the OWS results. Sphinxsearch then provides a singular position to search from, maybe using the ambitious SE Search Interface.

Certainly it won’t be as powerful as a Google solution, but it should do a sufficient job given the budget. I’ve called it Kerra to satisfy my penchant for cryptic names. The clue is ‘debian’, good luck figuring it out :) Anyone interested in packaging this all up with a base OS and releasing it as an Open Source Search Appliance?

I will likely be building a proof of concept of this over the next couple of weeks, will post results if I get anything substantial.

Article Tags>> | | |

iMac PC: revisiting an old friend

August 24th, 2008

Disturbed – Bean Bag

Sometimes I’ll start out a project, get the majority of the way through it and then dedicate my time to something else. I often don’t 100% complete my personal projects, usually because I have a combination of many projects, attention deficit disorder, liquor and whores, oh.. and actual work projects that I’m paid to complete. Over a year ago I was hacking a Via Epia motherboard and 15″ LCD into a G3 iMac case, but I never really provided much in the way of updates.

Well. The first challenge was the CDRom. The one I had was dead, and it used the proprietary Mac IDE interface. I got one from an orange iMac on Trademe for $15 and swapped the bezels, so that the colours matched. Then I discovered that underneath the Mac cdrom interface board was a very standard 2.5″ optical drive interface. So I found the appropriate part on Trademe again for about $7 and that was it – I had the iMac cd-rom working on a standard IDE controller! It does grind because it’s a very old drive, but it gets there in the end.

Then there was power. I hacked a spare PSU so that its AC connector lined up perfectly with the original AC port. The LCD is powered off this, and it passes a stable 12V signal to the picoPSU that is in turn powering the motherboard. The connection between the two PSU’s is unpluggable, meaning that the system module can be removed, just like the real thing. This also means that the LCD is always on – no need to worry about the LCD power button. If there’s AC, the LCD will sleep until it’s told otherwise, meaning the front power button only has to deal with the motherboard. Press the power button, the motherboard fires up and the LCD detects a VGA signal and follows suit. No electronics trickery required :)

And that was about it for the last year. Today I dealt to the next big outstanding task: getting the built in speakers working. This was so damn easy that I should have just done it ages ago. Basically, I found a pair of standard PC Speakers that ran on 12V, cracked them open and took out the amp and wiring. I found a piece of chassis where the buttons/dials wouldn’t get in the way and drilled the appropriate holes before mounting the amp.

I then soldered the left and right speaker cables, then the power. After double checking the soldering and heatshrink, I connected the disassembled iMac to mains and checked that the amp fired up. Then I plugged it into the lineout of my computer and switched through a variety of tunes while I adjusted the volume and tone knobs to reduce distortion etc, with a second set of ears nearby for a second opinion (cheers Penny Martins!). Once done, I reassembled the iMac, and now it’s sitting on the kitchen bench waiting for its new life as a kitchen box.

To do:
* Wireless. I’ll use the internal USB headers for this I think
* Touchscreen modification. In the kitchen is dandy, but the keyboard and mighty mouse take up some space. I think it’ll be used for a bit of jukeboxing, so making it touchscreen capable makes sense, and it’s a relatively cheap mod.

Maybe I’ll do if I can be bothered figuring it out:
* Work out the pin mapping for the front audio headers and connect these to the faudio block on the motherboard
* Work out the pin mapping for the IrDA device and connect that to the IrDA block on the motherboard
* Work out the built in microphone and wire that up

Article Tags>> | |

DIY Envelope Wax Seal

August 21st, 2008

Powderfinger – Private Man

This shit is hard! Basically I sliced a potato in half, then drew my work’s logo on it backwards, cut out the appropriate parts while melting a candle in a small pot.

You don’t need that much wax.. maybe a tablespoon’s worth. And let it cool down in the pot first, if you pour it too soon while it’s seperated, lots of oil will seep through the envelope and make a mess. Apart from that, the only other recommendation I can make is to practice your technique with one envelope – if it fails, simply scrape the wax back into the pot and re-melt it. Once you think you’ve got the technique down pat, do it on a fresh envelope :)

Why am I doing this? As a proof of concept for sprucing up the default login screen for the Zimbra project at work, and to let out my inner artist :)

Article Tags>> | |

Zimbra and external GAL sync’ing in Thunderbird

August 15th, 2008

Gnarls Barkley – Smiley Faces

Zimbra is targetted at a mix of web-based and corporate clients such as Outlook and mobile devices. Standards-compliant email clients and personal information managers (PIMs) don’t have native connectors but can hook up in standard ways such as IMAP or POP.

Two of the big features that Zimbra brings to the table, however, is an LDAP based Global Address List (GAL) and Calendaring.

As far as Calendaring in Thunderbird goes, install the Lightning plugin and configure it. As far as getting contacts from the LDAP directory goes… well… you could setup Thunderbird to poll LDAP directly, but that’s a bit involved for joe normal user. And what about when Joe Average is offline but wanting to queue up some emails in their outbox? How do they poll LDAP then? And isn’t this causing a bit of needless polling on the LDAP server?

The Outlook connector routinely syncs the GAL, presumably with a local copy. This makes offline addressing possible while reducing load on the LDAP directory.

Zindus achieves the same goal in Thunderbird. It’s quick to setup, and quicker to configure, and after the initial sync it’s very quick. No need for longwinded LDAP configuration instructions, no need for exporting and importing .csv’s, just results. This is the kind of user-friendly extension that Zimbra needs to advertise to promote greater integration for Thunderbird.

Article Tags>>

Disqus

August 13th, 2008

I just setup Disqus, if that’s your thing :)

http://rawiriblundell.disqus.com

Using Zimbra Desktop and a Proxy Server

August 13th, 2008

Paul McLaney – Don’t Wanna Know

I’m using Zimbra Desktop on KDE4.1 as Thunderbird seems to be a little crash-happy, and it also seems appropriate that I jump right into all things Zimbra if I’m going to be working on a Zimbra project. Zimbra Desktop is currently at Beta-3 and there seems no obvious way to configure it to load images via a proxy server, and Googling gives no relevant results.

Let me be the first to say “haha, bow at my feet!” for I have the fix. The above image is an html email with remote images loaded via a proxy server, and I will let you know the fix for one million dollars!

Any takers? No?

OK, the secret is that Zimbra is based on Mozilla’s Prism technology, so all you have to do is configure prism for a proxy configuration. Firstly, find all.js

rblundel@rawiri-desktop:~/zimbra/zdesktop/linux/prism/xulrunner/greprefs$ locate prism | grep all.js
/home/rblundel/zimbra/zdesktop/linux/prism/extensions/refractor@developer.mozilla.org/modules/WebAppInstall.jsm
/home/rblundel/zimbra/zdesktop/linux/prism/extensions/refractor@developer.mozilla.org/prism/modules/WebAppInstall.jsm
/home/rblundel/zimbra/zdesktop/linux/prism/modules/WebAppInstall.jsm
/home/rblundel/zimbra/zdesktop/linux/prism/xulrunner/components/nsXULAppInstall.js
/home/rblundel/zimbra/zdesktop/linux/prism/xulrunner/greprefs/all.js
/home/rblundel/zimbra/zdesktop/linux/prism/xulrunner/greprefs/xpinstall.js
rblundel@rawiri-desktop:~/zimbra/zdesktop/linux/prism/xulrunner/greprefs$

/home/rblundel/zimbra/zdesktop/linux/prism/xulrunner/greprefs/all.js is the one we’re after. This will be in a different location on Windows and Mac, but look in your Profile directory for a Zimbra directory, and simply go from there.

Edit all.js in your editor of choice, looking for

pref("network.proxy.type", 5);
//@line 798 "/builds/tinderbox/Xr-Mozilla1.9-Release/Linux_2.6.18-53.1.13.el5_Depend/mozilla/modules/libpref/src/init/all.js"
pref("network.proxy.ftp", "");
pref("network.proxy.ftp_port", 0);
pref("network.proxy.gopher", "");
pref("network.proxy.gopher_port", 0);
pref("network.proxy.http", "");
pref("network.proxy.http_port", 0);
pref("network.proxy.ssl", "");
pref("network.proxy.ssl_port", 0);
pref("network.proxy.socks", "");
pref("network.proxy.socks_port", 0);
pref("network.proxy.socks_version", 5);
pref("network.proxy.socks_remote_dns", false);

And edit to suit, as an example, for me:


pref("network.proxy.type", 1);
//@line 798 "/builds/tinderbox/Xr-Mozilla1.9-Release/Linux_2.6.18-53.1.13.el5_Depend/mozilla/modules/libpref/src/init/all.js"
pref("network.proxy.ftp", "");
pref("network.proxy.ftp_port", 0);
pref("network.proxy.gopher", "");
pref("network.proxy.gopher_port", 0);
pref("network.proxy.http", "192.168.0.1");
pref("network.proxy.http_port", 8080);
pref("network.proxy.ssl", "192.168.0.1");
pref("network.proxy.ssl_port", 8080);
pref("network.proxy.socks", "");
pref("network.proxy.socks_port", 0);
pref("network.proxy.socks_version", 5);
pref("network.proxy.socks_remote_dns", false);

That’s it. Restart Zimbra Desktop for prosperity and enjoy proxified goodness.

Alternatively, if your proxy has a .pac file, you can change this line:
// The PAC file to load. Ignored unless network.proxy.type is 2.
pref("network.proxy.autoconfig_url", "");

to something like
pref("network.proxy.autoconfig_url", "192.168.0.1:/some/directory/proxy.pac");

And obviously set pref(“network.proxy.type”, 1); to 2.

Note 1: My proxy server does not require authentication, so I don’t know how to configure Prism for authentication. There don’t seem to be any readily apparent options, but now that you know that the configuration is in the underlying prism platform, you’ve got something to google on.

Note 2: This is an unsupported hack. I won’t help you, the Zimbra support guys may or may not

Article Tags>> |

An unexpected loss

August 10th, 2008

Roy Orbison – Crying

Weasel loved shoulder surfing

My cat Wesley was put down. He looked so peaceful in the box the vets returned him in. My parents house is eerie now, with no Duke and no Weasel playing good pet/bad pet with your bloodpressure.

I never thought it would go down like this – I’d always imagined that the kitten I got 11 years ago because the pattern of his fur matched my basketball shoes would be put down, several years from now, while I held him in my arms and said goodbye. Instead he was alone at the vets in Levin, and I was stuck in Wellington :(

Article Tags>>

The latest project: Zimbra

August 6th, 2008

Lemon Jelly – Ramblin’ Man

I have been quite busy lately working with the Zimbra platform – trying to figure out a path to upgrade our existing infrastructure at work (OpenLDAP, Postfix, Mailman, SquirrelMail, SpamAssassin, Amavis, Mailzu etc) and the more I look at it, learn about it and work with it, the more I believe that it’s do-able. And the more excited I get about Zimbra as I learn about the capabilities that aren’t on the tin.

Our current setup has three DMZ’d MTA mail relays, which is our edge platform with SpamAssassin, Amavis, Mailzu and ClamAV. Ideally we would keep these in place, as there’s years of Antispam training there, and we’re this close to having Mailzu integrated into our Drupal intranet.

Therefore, we’d be running Zimbra in a pooled mailhosts configuration. The main issue then is integration with our existing OpenLDAP directory. There are two main schools of thought about how to do this:

1) Keep the directories seperate and map changes between the two with a cron’d script, or use filtered replication in syncrepl
2) Merge the directories, running an OpenLDAP based master with the Zimbra mailhosts running as slaves.

Personally I’m in favour of Merging the directories, however this might require some tweaks to the Zimbra LDAP configuration – and who knows what might happen as a result. This shouldn’t be overly hard though: Make sure the standard OpenLDAP schemas are the same, load in the Zimbra schemas and profit. The differences between standard OpenLDAP and Zimbra OpenLDAP are so far apart that the two could be merged into the same directory with little fuss – Zimbra essentially sticks to its own branch, and with the Zimbra schemas it’s a matter of mapping/replicating attributes from one branch to another.

More reading here and here.

There is a potential third option in MMR with syncrepl that should possibly be investigated.

There’s some exciting RFE’s in the Zimbra bugzilla db though, including this one, where marking a message as junk in your mail client will automagically train SpamAssassin, there is a cron script workaround too. This is excellent because it no longer requires the user to manually forward spam to spam.sldkjslkfdslfk@yourdomain.com – it’s completely transparent to the end user, which is how systems should be IMHO. My interest, however, is in how to replicate the SpamAssassin training between our existing MTA layer and a potential Zimbra mailhost layer. So anything that gets through our current antispam is marked at the user level, the mailhosts learn and this replicates upwards to the MTA’s – all transparent to the user!

Needless to say, this is a pretty fun challenge :)

Article Tags>> | |

Another mouth to feed

August 4th, 2008

His name is Nermel, and he’s a Devon Rex. He’s my older brother’s cat, he’s going to kick it with Uncle Ra because my brother is moving to Australia.

Article Tags>> | |