never let be - Home

propaganda

2010-02-26T20:55:00Z

So I have this formatting engine. Engine is maybe a strong word. I had a set of stylesheets and a nice ruby script I had hacked together that could convert HTML to XSL::FO, or XML Style Language, Formatting Objects. FO is one of those technologies that, given CSS seems a bit overkill, but from a layout perspective is probably vital. The really nice thing is, some smart guys started the Apache FOP project which allows you to render a set of formatting objects to various outputs, for example PDF.

So how do you get an FO document? Well, you could write on by hand but that is what the crazy people do. I didn't want to be crazy so I looked around for an XSLT stylesheet that would transform HTML to FO. There are a couple in the wild that do this. Notably there are a set of document from Antenna House that covered almost everything. The problem was that they worked with their formatting engine, and not Apache FOP. So I spent some time a year ago reworking them so that they played well together.

I had all of these pieces sitting around and wanted to cobble them together, but I rarely use HTML anymore. I use markdown or textile. So I decided I needed to hack together something that was easy to install to convert those to HTML and then generate a PDF for emailing to people. A couple of hours hacking gave me Propaganda. There is support for shelling out to the Java command for the Jar and support for loading the JVM directly with the Ruby Java Bridge. It can detect what you have and go for it.

lego star wars at-st walker

2010-01-19T18:27:00Z

So Jude and I play with Lego blocks a lot. He loves it. That's why I do it. Not because I am a giant kid, or geek, or nerd. Or maybe I am. In any event, he has been asking for Lego 'The Battle of Endor for a while now, and we have been resisting. Today he decided he wanted to build it with what he had. So we thought about it a little bit and started in and I think we will be able to do most of it. We worked on it for about an hour this morning. Herein are a couple of shots of our first draft at the AT-ST walker:

The middle guns need some work still, and we need to add the hatch for Chewbacca (and reconstruct the inside a bit to firm it up), but the legs seem to work great and he is already happy.

pih, baobab, openmrs, x-forwarding, mateme and you

2010-01-15T05:01:00Z

So I needed to test some stuff locally on a physical touchscreen (the kind that we use in Malawi). This required some setup that I haven’t needed to do in a while and may not have to do ever again. So I am capturing it here for posterity I guess.

The goal

Baobab style installations have multiple touchscreens (iOpeners running Baobab Linux) that are powered over ethernet (POE)`. The power is injected at a 48V POE switch, meaning the Ethernet cable carries both power and data. A safe port on the switch allows you to plug in a server.

The iOpeners are very lower power and very low capability. Because of this we opted for one of the great all-time hacks (suggested by MERL and implemented by toggles_w) of running all of the firefox instances on the server and x-forwarding them to the touchscreens. This brought about much more crazy hacks (the reverse netcat label printing hack comes to mind); but sadly those are out of the scope of this article.

Now, the switch is kind of unimportant except for injecting the power. In my case I haven’t got anything that fancy, but when you know Gerry you end up with cool things and I just so happen to have a POE injector that just handles adding power to a line that carries data. So I have plugged that into a run-of-the-mill wireless router.

Plugged into the router I have an Ubuntu machine that will act like my server. This is just a Toshiba laptop running Ubuntu 9.04.

The wires

Do not fry your router by plugging things in backward. We are dealing with power that kills things (mainly computers) here. If you aren’t sure what you are doing then just stop. If you are kinda sure, you can usually just look at the heads of the connectors, the side with more wires in the RJ-45 has power. Plug the hot end into the touchscreen (which I know you all have modified by adding a daughter board which takes power and does other cool stuff). Plug the non-hot end into a LAN port on the router. Plug in the injector. At this point you should be able to power on the touchscreen and it should get an IP address on your router’s network.

Setting up the router

I don’t remember why, but the server needs to be in charge of handing out addresses. By default the router handles DHCP and we need to turn that off. In Ubuntu, go to the Network Manager and make sure everything is set to the default (DHCP/automatic) for the LAN connection and plug in. Then you can just manage your router through its web interface (mine is at 192.168.0.1, which I changed to 192.168.1.1). Find and turn off DHCP on the router, and reboot it.

Now you got nothing

At this point you have no server, no addresses. You have a lot of wires that won’t do anything. So lets make them do something. You will need to install lots of stuff on your Ubuntu computer. Maybe Mike will make a nice live boot USB stick that does all of this someday. That would be cool.

Install, install, install

First you need to add some packages. You may already have some of these, but just in case:

sudo apt-get install build-essential dnsmasq xinit xauth

Next, create a user for the daemons (you can use whatever password you want):

sudo adduser firefox
sudo su firefox
cd ~

You’ll need to check out the daemon code and compile it:

git init
git remote add origin git://github.com/baobab/firefox-daemon.git
git pull origin master
cd src
make
cp apps/baobab.firefox/baobab.firefox* ..
exit

Now lets make it so that the daemon has a controller and runs at startup:

sudo cp /home/firefox/firefox-daemon /etc/init.d/
sudo /usr/sbin/update-rc.d -f firefox-daemon defaults

Setup your server to be more networky

So one of the things you need is to handle DNS using dnsmasq which governs the range of numbers DHCP will hand out. Delete everything from your /etc/dnsmasq.conf file, and make it look like this (replace x with your subnet, 1 in my case):

dhcp-range=192.168.x.100,192.168.x.150,12h

Then you want to remove everything from your /etc/resolv.conf. Now this might have other side-effects, I am not sure. Regardless it will make the touchscreen boot very very slowly if you skip it. Everyone will complain.

Make your /etc/network/interfaces look like this (again replace x with your subnet):

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
address 192.168.x.200
netmask 255.255.255.0
gateway 192.168.x.1

Now, in Baobab Linux there is one requirement, it needs to telnet to “bart”. We didn’t want to have a server address in there because every network is going to be different, so we needed to let the server resolve some name. “bart” or “Baobab Anti-retroviral Treatment” software is what we made this whole thing for, so the touchscreen tries to connect to a “bart” server. It doesn’t actually need to have the software installed, it just needs the name.

Dnsmasq has a nice trick of resolving names based on the server’s /etc/hosts file. So edit that and add the following:

192.168.1.200 bart

Where ”.1” is your subnet. Once you have all this you can reboot your network with

sudo /etc/init.d/networking restart

Then restart dnsmasq:

sudo /etc/init.d/dnsmasq restart

Installing the firefox-2 binary

X-forwarding does not work very well with Firefox 3. There are some options you can tweak in your X11 config to make it better but I haven’t explored it fully. It does, however, work very well with Firefox 2. So you need to install that. But it isn’t supported anymore. Like at all. In fact finding it and installing it on Ubuntu 9.04 was a challenge, initially. You can just follow these instructions:

sudo apt-get install libstdc++5 libmozjs0d
mv ~/.profile ~/.profile_bak
wget ftp://ftp.mozilla.org/pub/mozilla.org/firefox/releases/2.0.0.20/linux-i686/en-US/firefox-  2.0.0.20.tar.gz
tar xvzf firefox-2.0.0.20.tar.gz
sudo mv firefox /usr/bin/firefox-2.0
sudo rm /usr/bin/firefox
sudo ln -s /usr/bin/firefox-2.0/firefox /usr/bin/firefox

Connecting the touchscreen to do something

At this point you should be fully setup. Go ahead and run the daemon so you can see any connections and any output:

sudo /etc/init.d/firefox-daemon

You should see:

Connected to port 5840

Plug in the touchscreen and turn it on. It should boot, give you a couple of messages in X and then startup a Firefox session.

h3 Bonus points

Now the point of this is to point it at Mateme, the new touchscreen based EHR we are using in Malawi. But you might want to do something else. Simply modify the /home/firefox/default.profile/prefs.js file and change the following:

user_pref("browser.startup.homepage", "http://localhost");

To point at whatever you want. Keep in mind, that to get out of the network you are on you will probably need a dual network card (i.e., LAN/wifi).

Here is a shot of everything working:

some background on windows 7 starter

2010-01-11T17:01:00Z

So you can't change the desktop background on Windows 7 Starter edition. Really. Well, you "can't" is more accurate. Thanks to some nice comments from Gabe and srg84 here I now can.

Quoting directly:

Step by Step:

Open regedit (aka registry editor; you can access it from the star menu by typing “regedit” in the search box…)
Go to “HKEYCURRENTUSER\Control Panel\Desktop\” folder and click on it
On the right hand side find there are a bunch of entries. Find the key named “Wallpaper” and double click on it and put the path of the picture you want as your new wallpaper (example path is “C:\Users\Bob\Pictures\new_wallpaper.jpg”)
Right-click on the “Desktop” folder in regedit that you found in step 2 and click Permissions.
Click “Advanced”
Go to “Owner” tab, highlight your name in the box that says ‘Change owner to’ ( There are only two choices the other is Administrator )… once your user name is highlighted click “OK”
Click on “Advanced” again
Uncheck the button that reads “Include inheritable permissions from the object’s parent”… click “Remove” when prompted
Click “Add”
Type “Everyone” and click “OK”
Check Allow “Read Control” and click “OK”
Click “OK” again
Highlight “Everyone” and check to Allow “Read” and click “OK”
Restart Computer and enjoy new Wallpaper

Basically what the steps have done is remove the ability to do anything but “Read” that piece of info. That means modifying that piece of info is not possible as well… unless you change those permissions. So….

If you want to change the wallpaper again after you have gone through the steps above once just do the following:

Open regedit
Go to “HKEYCURRENTUSER\Control Panel\Desktop\” folder and RIGHT-CLICK on it, and go to “Permissions” (just like step 4 above)
Check the box to Allow “Full Control” and click OK
Now you can go to “Wallpaper” on the right hand side and change the path to the wallpaper you really wanted
Right-Click “Desktop” folder again and go to “Permissions”
UN-CHECK the box that says Allow “Full Control” and click OK
Go to Start Menu, Log Off —-> Log On again… and you should have a new wallpaper again

Or just install Ubuntu.

meaningful use

2009-12-31T08:17:00Z

So the Health and Human Services department released an interim final rule for certification of electronic health record (EHR) technology here in the States (http://healthit.hhs.gov/blog/onc/index.php/2009/12/30/a-defining-moment-for-meaningful-use/. The crux of this is that in the new recovery bills a lot of money is targeted at expanding the use of helath information technology (HIT). In order to get this money, you have to be an “eligible provider” with “meaningful use” of a “certified EHR”. When the legislation was written, the last two qualifications were pretty vague. Over the past few months, “meaningful use” has a set of criteria and a timeline (which includes point of care in 2013!). The final piece of the puzzle took shape today:

The certification criteria adopted in this initial set establish the capabilities and related standards that certified electronic health record (EHR) technology will need to include in order to, at a minimum, support the achievement of the proposed meaningful use Stage 1 (beginning in 2011) by eligible professionals and eligible hospitals under the Medicare and Medicaid EHR Incentive Programs.

The document (PDF) is 136 pages, which is a bit overwhelming. However, there is a lot of extremely useful history and basis for the new rule that is included making it a useful read if you aren’t familiar with the plethora of acronyms that come with health information technology (e.g., HIPPA, HITECH, PHSA, CCHIT, AHIC, HITSP).

The rule is interesting to me on a number of levels, not the least of which is its relation to the National Data Standards work we did in Malawi. It is fascinating to see Malawi so far ahead on so many of the salient points. Additionally, the challenges from an IT perspective are strikingly similar (harmonizing code sets, agreeing on data transfer protocols, etc). I wonder if this is because the problems are germane to medical informatics or because we applied too much Western thought to the problem at hand.

Ultimately, to achieve semantic interoperability, we anticipate that multiple layers – network transportation protocols, data and services descriptions, information models, and vocabularies and code sets – will need to be standardized and/or harmonized to produce an inclusive, consistent representation of the interoperability requirements. We anticipate using a harmonization process that will integrate different representations of health care information into a consistent representation and maintain and update that consistent representation over time. For an information model, this process could include merging related concepts, adding new concepts, and mapping concepts from one representation of health care information to another. Similar processes to support standardization of data and services descriptions and vocabularies and codes sets may also be needed. (pg 31).

Still, I can’t help but think that we should admit defeat on shared code sets and start solving the problem of the comparability of heterogeneous, unstructured information.

Check out pages 51-61 for Table 1A (the main criteria) and Pages 79-81 for Table 2A (the content and vocabulary criteria standards) and Page 85 for Table 2B (the security criteria standards). Jumping through the tables you get the gist of what you need to know pretty quickly.

Simultaneously the Electronic Health Record Incentive Program for Medicare and Medicaid was released. I haven’t started on that one yet (it is 556 pages long).

syntax highlighting more files in xcode

2009-11-24T18:52:00Z

So I use XCode. I really do. And I kinda like it. But one thing I hate is that there are files it doesn’t want to highlight as Ruby. This has been discussed elsewhere, but I needed to be able to find it more quickly. Here is how you change that. You need to edit the file: /Developer/Library/PrivateFrameworks/DevToolsCore.framework/Versions/A/Resources/Standard file types.pbfilespec. You want to find these two lines:

Extensions = (rb,rbw);
MagicWord = ("#!/bin/ruby", "#! /bin/ruby", "#!/usr/bin/ruby", "#! /usr/bin/ruby", "#!/usr/local/bin/ruby", "#! /usr/local/bin/ruby");

And change them to:

Extensions = (rb,rbw,rake);
MagicWord = ("#!/bin/ruby", "#! /bin/ruby", "#!/usr/bin/ruby", "#! /usr/bin/ruby", "#!/usr/local/bin/ruby", "#! /usr/local/bin/ruby", "#!/usr/bin/env ruby");

Also, find:

Extensions = (shtml, jsp, rhtml);

And change it to:

Extensions = (shtml, jsp, rhtml, erb);

compiling festival lite on osx

2009-10-12T21:45:00Z

Festival is a text-to-speech (TTS) system that was developed until 2005 by the Speech Group at Carnegie Mellon University It is a nifty little tool and the voice sounds like a robot. Jude and I have been working on our robot and will need some text to speech capability on our OpenWRT Asus WL-520gu. Of course, I also want to test things on my Mac and therefore I needed to complile the festival lite kit locally. This took some digging, but the mailing list archives turned up the answers.

First you will want to go to the website to see whats there:

http://www.speech.cs.cmu.edu/flite/

Then in a terminal grab the source:

wget http://www.speech.cs.cmu.edu/flite/packed/flite-1.3/flite-1.3-release.tar.gz

Unpack the source and go to the folder:

tar xvzf flite-1.3-release.tar.gz
rm flite-1.3-release.tar.gz
cd flite-1.3-release

Now here is the trick. The Apple sting.h file defines a method called swab which we don’t want. So lets cheat and make a copy of the string.h file in our local include folder:

cp /usr/include/string.h include/
chmod +w include/string.h

Now, comment out the offending line (for me this was line 137):

// void     swab(const void * __restrict, void * __restrict, ssize_t);

Then in your terminal, make:

make

At this point everything should build (assuming you have a good build chain). Then you can run flite:

bin/flite --help

Of course, some of the options are not going to work (namely, outputting to an OSX sound device). For me that is okay, I can just process it through lame and play it using madplay or mpg123:

bin/flite "Hello World" hello.wav; lame hello.wav; mpg123 hello.wav.mp3

ant based testing for openmrs

2009-09-01T00:17:00Z

So I needed to do some module development on OpenMRS and I didn’t want to setup eclipse. I had already figured out the basics of gabbing the code, building, and deploying things… but testing was missing. So I spent some time playing with JUnit ant tasks. In the end I came up with a single build.xml file that will work for core and for modules:

<?xml version="1.0"?>
<!-- ********************************************************* -->
<!-- ** Tests                                               ** -->
<!-- **                                                     ** -->
<!-- ** Some options:                                       ** -->
<!-- **                                                     ** -->
<!-- **   ant test -Dtest=org.openmrs.YourClassTest         ** -->
<!-- **   ant test -Dformat=(plain|brief|xml)               ** -->
<!-- **                                                     ** -->
<!-- ** @author Jeff Rafter                                 ** -->
<!-- ** @version 0.1                                        ** -->
<!-- ********************************************************* -->
<project name="test" default="test">    
  <property name="fork" value="yes" />
  <property name="base.dir" location=".." />
  
  <condition property="format.plain">
    <matches string="${format}" pattern="plain" />
  </condition>
  
  <condition property="format.brief">
    <matches string="${format}" pattern="brief" />
  </condition>
  
  <condition property="format.xml">
    <matches string="${format}" pattern="xml" />
  </condition>
  
  <path id="classpath">
    <pathelement location="api"/>
    <pathelement location="web"/>
    <pathelement location="build"/>
    <pathelement location="${base.dir}/build"/>
    <pathelement location="${base.dir}/dist"/>
    <pathelement location="${base.dir}/web/WEB-INF"/>
    <pathelement location="${base.dir}/web/META-INF"/>
    <!-- This might be a little greedy -->
    <fileset dir="${base.dir}">
      <include name="**/*.jar" />
    </fileset>
  </path>    

  <target name="test">
    <!-- Clean out the old test class build area -->
    <delete dir="build" />

    <!-- Setup the new test target -->
    <mkdir dir="build" />

    <!-- Copy over the log4j.xml so that it is used -->
    <copy todir="build/">
      <fileset dir="metadata/" includes="**/*" />
    </copy>

    <!-- Compile the tests -->
    <javac destdir="build" debug="true" debuglevel="lines,source,vars">
      <classpath refid="classpath" />
      <src path="." />
      <include name="**/*.java" />
    </javac>

    <!-- Run the units -->
    <junit printsummary="true" haltonfailure="false" failureProperty="test.failure">
      <formatter type="plain" usefile="false" if="format.plain"/>
      <formatter type="brief" usefile="false" if="format.brief"/>
      <formatter type="xml" usefile="false" if="format.xml"/>
      <classpath refid="classpath" />
      <test name="${test}" if="test"/> 
      <batchtest fork="${fork}" unless="test">
        <fileset dir="build">
          <include name="**/*Test.class"/>
          <exclude name="**/*BaseTest*" />
          <exclude name="**/Base*" />          
          <exclude name="**/OpenmrsTestsTest*" />
        </fileset>
      </batchtest>
    </junit>
    
    <!-- Fail the ant build if a test failed -->
    <fail message="test failed" if="test.failure" />
  </target>
</project>

I put this file in the test folder in the repository. This means that it is separate from the main build.xml which I felt was a bit safer for now. To make the output behave, you need to create a separate log4j.xml to silence a lot of the output. Create a metadata folder inside of your test folder and make copy the following content into log4j.xml:

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
  <appender name="CONSOLE" class="org.apache.log4j.ConsoleAppender">
    <layout class="org.apache.log4j.PatternLayout">
      <param name="ConversionPattern" 
        value="%p - %C{1}.%M(%L) |%d{ISO8601}| %m%n" />
    </layout>
  </appender>
  <root> 
    <priority value="FATAL" /> 
    <appender-ref ref="CONSOLE" /> 
  </root>
</log4j:configuration>

If, for some reason, you need the logger output, simply change the FATAL value to DEBUG or WARN.

Also take note that I have added a couple of options to the main task:

12	<!-- ant test -Dtest=org.openmrs.YourClassTest --><!-- ant test -Dformat=(plain\|brief\|xml) -->

Passing in the test property allows you to run only the tests in a single test class. It still recompiles all of the tests, but that doesn’t seem to take too long. JUnit allows multiple formatters, but by default I am not using them. If you want to see more verbose output set the format property.

One last option:

1	<property name="fork" value="yes" />

I have broken out the option to fork when running the tests. By default the test runner will fork a new JVM process for every test class. Turning this off makes the tests run much much more quickly. In module tests this will generally work great, however, turning this off might run you out of memory, and breaks the core tests. To turn it off, simply change the value to no.

suggesting prescriptions

2009-07-31T13:09:00Z

So it has been a busy month in Malawi. Seriously busy. Lots of code written, lots of new features and in general cool cool things. While in Lilongwe this past month, Mwatha, Kondwani, Soyapi, Dave and Edmond helped me merge the Partners in Health and Baobab Health versions of our touchscreen electronic medical record systems. We had forked the project as Baobab started on the its off-the-grid Ministry of Health sites and as we in Neno started focusing on outpatient diagnosis and treatment. This is a long way of saying that Baobab had made some cool stuff and PIH had made some cool stuff and we put it together to make everyone a little more cool.

One of the features that has me most excited is patient prescription suggestion.

Treatment

The treatment workflow will generally be performed by clinicians after a diagnosis has been made. The user selects this workflow (or it is automatically selected) and they are first asked for which diagnosis the treatment is being made (no diagnosis is an allowable answer).

If a diagnosis is chosen then the system, then the system will suggest popular treatment protocols for that diagnosis based on historical information collected in the system. This is where the feature really comes alive. Basically, because every prescription is connected to a specific diagnosis we can look in the database and see what the most popular prescription for the patient's current diagnosis is. Now, this may not always be right (more on that later) but it will cover the vast majority of cases and truly speed up the workflow in the clinic.

If the user selects a prescription, it is added to the patient's list of orders and will be printed on a visit label that can be presented at the pharmacy.

In some cases (especially when the system is first deployed and not full of historical data), it is possible that the user will not want to use this and will instead choose to create a new prescription.

When building a prescription the user must first choose the generic drug and then a filtered list of formulations of that drug. We actually worked on several revisions of this and landed here. Basically we found that there were so many forms of similar drugs that we needed to do it in two steps. This way you can pick the kind of drug (for example Ciprofloxicin, and then the specific formulation and strength.

From there two types of prescription are allowed "Standard" and "Variable". A standard prescription means that each dose that is given is the same (for example 2 tablets in the morning, 2 tablets in the evening). The user should choose "Variable" when the dose strength is different at different times in the day (for example: 1 tablet in the morning 1/2 tablet in the afternoon, 2 tablets at night). We find these to be far less common but they do occur (especially when prescribing starter packs of drugs for Antiretroviral treatments).

The dose strength allows the user to input how much (in specified units that the drug is delivered in) each dose should be. So in the case of something like Cipro the drug is dosed in mg, however in other drugs it might use "tablets" or "vials" or "mg/mL" for the units. This is configurable in the database when drugs are added.

Next you need to decide the frequency for the dosage. This comes from a set of concepts in the database and is ultimately stored as plain text. Really, I wish the frequency were stored as a concept and concept name (in OpenMRS speak) because then it could be aggregated a little better and faster, but this works okay. If you were prescribing a variable dose this would be skipped and instead you would need to select the dose for each period (in the morning, in the afternoon, in the evening, at night).

Finally you select the number of days to continue the prescription and whether or not it should be taken only as needed (e.g., for pain).

New prescriptions are listed in the prescription dashboard. Tracking which orders are filled (dispensed) is not currently part of the treatment workflow (as we do not currently have a pharmacy workstation in Neno).

But wait, it might be wrong!

Right. It might not have the best prescription for a particular patient or even a particular diagnosis. We are conscious that it doesn't give any advice about dosage strengths for age or weight bands and that this might be problematic. This slowed us down for a long time until we realized that the system is not supposed to replace clinicians. It is supposed to make them faster. It keeps the clinicians from needing to write anything in a large number of cases which makes the data more consistent, auditable, and readable by the pharmacist. But in general you need someone with clinical experience to know if the suggestions are right or wrong. I look forward to checking the data in a few months to see how often non-suggested prescriptions occur.

Background

Now how this feature came to be is one of those great mysteries of "whose idea was that?" that often happen when you have a really great project manager. I mean, it is pretty common that people want medical record systems to do magic things with data, so the idea is not necessarily novel. Nevertheless, the actual data and workflow pieces of the implementation here came from a late-night/early-morning Skype session with Evan after he had been meeting with clinicians. Evan Waters fills the role of project manager (among other things) in Neno and is one of those truly brilliant guys. He is probably more socially brilliant than technically (where he is quite adept) and he just knows how to get the best from people.

A lot of the thinking on how prescriptions should work on the touchscreen came from work that Gerry Douglas and Zach Landis Lewis had done for the pharmacy system at Kamuzu Central Hospital. Those evolved into the Baobab Anti-Retroviral treatment prescription system used in the HIV clinics that Baobab supports. All of this led to a series of meetings at Lighthouse where a group of us (including Andreas Jahn, Soyapi, and Mike) hashed out just how to use the drug tables in OpenMRS. Ultimately Evan and I took this, formulated a proposal to the OpenMRS developers list and simplified where we could.

With the one change of connecting the diagnosis observation to the drug order, this should be generally useful and possible inside of OpenMRS and should maybe become a module. Of course, this assumes that you have the data and the patient in the same place when you are prescribing and that the information for their current diagnosis is in the system. To do that you need to have a point-of-care system.

Want to try it?

As always you can go try out a demo copy at http://mateme.socialrange.org using the user name "Demo" and the password "Demo". Please don't input real patient information in there! Also, keep in mind the system is "empty" right now and I will probably update the database with the data learned from Neno soon.

the automatic flushing system

2009-06-10T21:15:00Z

So back at the University of Iowa, one of my favorite classes was by Jeff Porter. Radio Essays. Basically a step by step how-to for This American Life style audio essays. We all made them. It was great. There was a whole website that was devoted to it on the Twist server but it seems to have vanished. I will have to get it online here. In the meantime I uploaded my essays

The Automatic Flushing System

And

The story of my broken leg. Now I'm Old.

I really liked both of these when I made them, but realize now that I wish I had cleaned them up some more. I would be better at that now, I think. I also uploaded Nick Gregory's "Sailor Boy" here which I had around because I always thought it was the best one of the whole class. It's great I think. Hopefully he sees this and contacts me

Sailor Boy, by Nick Gregory

github-gem-build

2009-05-20T06:11:00Z

So the fact that GitHub hosts gems is really great. I find myself writing gems all the time now because it is easy to publish (and super easy to write thanks to jeweler). But the gem builder on GitHub is finicky. And when that's working, getting the gem into the spec file is slow. I hate waiting. I also hate building the gem locally because it always ends up without the owner's name prefixed. Then when the gem is ready on GitHub I end up with two different gems and a debugging nightmare.

Well, I got a little fed up and wrote a script that clones a repository, changes the gemspec so that the name matches, builds the gem and installs all with a simple command:

github-gem-build owner repository

It also validates the gemspec using the same code that GitHub recommends for testing if you are having problems. I probably need to tweak a few things to make it work properly on Windows (removing the sudo call if it is running on Windows, fixing the fact that File.join uses '/'), and I imagine that is coming soon. It might also be nice to support multiple gem versions based on tag or something. I packaged it as a gem so that you could just install it through the gem install command:

sudo gem install jeffrafter-github-gem-build

The irony? The gem builder didn't want to build it right away so I had to use it to install itself.

github has an apps platform

2009-05-14T20:13:00Z

So when Nick told me about his http://rdoc.info project for generating Ruby documentation with YARD it hit me immediately. Like a ton of bricks in the face. The place for documentation about GitHub projects is on GitHub. Why had they never built it in? Then I realized, they didn't need to. We could do it ourselves.

The concept was simple enough: register the user name "docs" on GitHub and create a GitHub Pages repository. This would allow us to publish content to http://docs.github.com. So I hacked around on a prototype (borrowing every image, stylesheet and layout I could from the exsiting GitHub site) and we came up with something that looked pretty good.

Uploaded with plasq's Skitch!

We realized though, that we would need to get permission from GitHub to do this before we showed anyone. So we took everything to Rails conference and met up with them over whiskey in the Star Trek lounge. They instantly got it, gave us the go ahead, and started talking about what would need to happen for it to really work well. We grabbed some beers, paired up with Joe and started hacking some more.

Uploaded with plasq's Skitch!

It turns out it's really hard to go back to static websites with no server-side code. But with GitHub pages you have to do just that. So we built a couple of clever JavaScript libraries for accessing repository and commit information from the API. We also made use of the 404.html page that you get with GitHub Pages. Add a couple of meta-redirects and one cross-site post and you have something that feels pretty normal.

Uploaded with plasq's Skitch!

When you search for a repository that has not been generated yet you are pushed to the 404.html where there is a form for generating. After entering the information, we check that the repository really exists and then post to http://rdoc.info. Everytime it generates documentation for a project it simultaneously generates pages using our custom GitHub YARD template and pushes it to the "docs" repository. We tell it that we want a redirect to our "generating" page so you don't notice that you left.

Uploaded with plasq's Skitch!

For the documentation itself, we maintain separate copies of the documentation per commit SHA. This allows you to have links to various versions of documentation which are static and permanent. For each page we added a namespaces button and methods button instead of using frames.

Uploaded with plasq's Skitch!

When you view source for a given method, there is a link back to the actual revision of the file in the repository. This allows you to quickly see specific source areas in context.

Uploaded with plasq's Skitch!

We are eager for feedback; please comment here or send us a message on GitHub.

no such file or directory

2009-05-09T21:48:00Z

Sorry, brief interlude here for something that keeps getting me. I get this error on getcwd all the time when doing programmatic git things. This is how you recreate it:

Make a directory, change to it and open a console:

cd /tmp
mkdir foo
cd foo
irb

Check the pwd:

$ irb
irb(main):001:0> Dir.pwd
=> "/private/tmp/foo"

Now, in a separate terminal, delete that foo folder (leaving irb open)

rm -rf /tmp/foo

Now, back in your console, get the pwd again:

irb(main):002:0> Dir.pwd
Errno::ENOENT: No such file or directory - getcwd
	from (irb):2:in `pwd'
	from (irb):2

So if you are getting this in your code, it is because you are deleting a folder out from under yourself (which tends to happen a lot when testing).

full text searching in javascript

2009-04-27T03:44:00Z

So I have this project I am working on with Nick. One of the things we needed was a fancy customizable search solution. Generally, this is done in the ruby world using something like Spinix, Ferret, Solr or the like. Only problem is that all of these require you to have some fancy server-side setup. But for our project we weren't going to have that-- only a static site that we generate once.

Well, I had a little search engine I had written called Fates search. It was based on Mauricio Fernandez's FT_Search (see http://eigenclass.org/hiki/simple+full+text+search+engine). With some help from Nick I cleaned up a lot of my code (it could use more cleanup, believe me). Then in a couple of late night fits of coding, I converted the whole thing to Javascript.

There was a problem. The reason the fulltext searching was quick was because of some elegant file seeking and binary searching. But if every user had to download megabytes and megabytes of indexes and the fulltext corpus every time they wanted to search... well, they would not be impressed. So I decided that I needed to shard the whole thing. Basically I divided the main index (the suffixes) into four-character shards. So hits for "Jeffrey" would fall into the "jeff" shard. Clearly, the right length depends on the data you are indexing, but for my fifty-thousand name sample, four seemed about right. Most of the index shards turned out to be less than 1kb with a couple of pathological cases for common names.

However, it wasn't just the main index that needed sharding. In order to run the binary search, each step needed to peek in the main fulltext corpus which contained all of the text, and which was of course, very big. Because the file is not ordered in any way, sharding by a prefix wasn't an option. But because I would have offsets into the file for seeking, I figured I could just chunk the whole thing into smaller 1kb file chunks. This way, if the search engine needed to seek to offset 15693, I could simply divide to find the appropriate file chunk, retrieve it, then seek to the position based on the modulus.

Finally, I added a little bit of jQuery goodness so that it would look prettier while you search.

I posted the code to GitHub at http://jeffrafter.github.com/fates and the Indexer is available in the main Fates repository http://github.com/jeffrafter/fates. Try it out and make sure you watch the console in Firebug to see the shards and chunks being retrieved.

the sensory web

2009-04-16T22:56:00Z

I know I keep talking about the next web and everyone is getting a little bored I am sure. But it is beginning to make sense to me, really. For a long time I have been talking about a very specific Wired article about augmenting human senses. Basically, some German researchers thought it would be interesting to give humans a sense of true north. Birds have this sense built in (maybe from magnetite in their stomachs?) and it helps them to migrate thousands of miles and know where they are going. But humans don't have magnetite in our stomachs (unless you are a cyborg. Its why men get lost and need to ask for directions.

So the Germans made a belt. And the northern most part of the belt would vibrate just a little. After a month, the bird-people had molded their brains enough to expect this. The researchers found that the people had an incredibly good sense o direction. In fact, when the experiment was over and the belts were taken away most of the people freaked out and bought GPS devices.

This is why people freak out when Twitter goes down. Twitter is a new human sense. Throughout history some people have been really good in social situations. They had an innate sense to say or do the right thing, or to better understand the people they were talking to. Clearly these people had a distinct advantage and we evolved into a social species.

The Futurists all wrote about how computers will think for us. And I they were probably very close. The computers will add senses to us. 6th senses. 7th senses. People will become more and more intuitive based on these enhanced senses. Mike tells me I should be reading Kurzweil and he is probably right.

The "Social Sense" that twitter adds is like the true-north belt. When you receive status updates it is like a little buzz telling you what is happening in your chosen social sphere. If this just happened once it would be helpful. Instead it happens constantly and your brain adapts to expect this information. For people who don't use Twitter (or Facebook status updates) it is difficult to understand this. It seems like exhibitionism and voyeurism.

But suppose you did it. Suppose you got status updates on your phone and you could tell from the latest updates that a particular friend was feeling down. This might cause you to call or see the friend to cheer them up. Critics of Twitter are always saying that you should call your friend to ask how they are doing. Sure. But what if you just knew?

More than once I have stopped myself from making an ill-timed joke or recognized the dynamic of a room of friends at a party simply because I follow their status updates. I admit it isn't perfect, but I feel it gives me a distinct advantage in social situations compared to those who do not understand the context.

This is why the notification web is important. This is why twitter is a platform. The interesting thing about sensory perception is that it is a series of notifications you sense things by letting your brain respond to incoming streams of sensation. Smells, lights, touch... anything. Notification applications (like Twitter) will tend toward success if they cultivate "senses".

Twitter focuses on social senses, but there are a lot of other lesser known senses that could be enhanced. When someone says "he has good business sense" it is a figure of speech. What they mean is, he has an intuitive ability to do the right thing in business situations. Suppose you can train the brain to respond better by using an enhanced set of notifications. Something more fundamental than just having good data. Something targeted and constant. Can you imagine if you had the same sensation about a business deal as you had standing on the edge of a cliff? We see the height of the cliff and know not to walk off of it, mainly because we have trained our minds by walking off of smaller things. People understood this before understanding gravity. Maybe it is possible to give someone good business sense without them understanding the dynamics of market forces and economics as well.

It is fairly amazing to watch the rate at which the internet is going through massive iterations.