Strip unwanted dates from QFX files

I can't imagine there are many people out there who are going to find this interesting, let alone useful, but here goes.

I use a software program to manage my finances. Once a month I download a list of transactions for the previous month from my bank and my credit cards. The websites for these institutions allow me to specify a date range, and then download a QFX file that contains all the transactions within that range.

Except for americanexpress.com. Their website only lets me specify a one month range that corresponds to my account's billing cycle.

So, for my half-dozen bank accounts, and my Visa credit card, I can download a QFX file that contains transactions from the beginning of the month to the end of the month. But for Amex I have to download two months of data, from the 9th of the month before, to the 8th of the month after. Or thereabouts.

Big pain in the ass when you're trying to import transactions quickly, because you have to go in and either match transactions or delete duplicates. Yes, I could make all the QFX files correspond to the Amex card's billing cycle, but will only work as long as all the other websites are flexible, and makes keeping a month to month budget more difficult.

So, I wrote a Perl script that will take a QFX file, remove transactions that fall outside a specified range, and then output the remaining transactions to a new file.

(I added a feature that will preserve transactions that fall too far outside of the specified range, in case something weird is going on with the institution's billing. This tolerance can be tuned.)

This script is barely tested, isn't annotated, and probably isn't the most efficient way to do the job. So I don't make any guarantees. If you are actually reading this, and actually want to use the script -- which is about as likely Paris Hilton becoming a nun, or Oprah getting down to a size 2 -- you do so at your own risk.

That said, thar she blows ... (the script, not Oprah) ...

#! /usr/bin/perl

use Time::Local;

$usage = "
USAGE: qfix.pl filename lowdate highdate
DATE FORMAT: MMDDYY
EXAMPLE: qfix file.qfx 010109 013109
";

$infile = $ARGV[0];
$lowdate = $ARGV[1];
$highdate = $ARGV[2];
$lowtol = 31 * 24 * 60 * 60; # days * hr/day * mins/hr * secs/min
$hightol = 30 * 24 * 60 * 60;
$outfile = "$infile.$lowdate-$highdate";

unless ($#ARGV == 2) { die "$usage\n"; };


$ARGV[1] =~ m|\A(\d{2})(\d{2})(\d{2})|;
$lowdate = timelocal(0,0,0,$2,${1}-1,"20$3");

$ARGV[2] =~ m|\A(\d{2})(\d{2})(\d{2})|;
$highdate = timelocal(0,0,0,$2,${1}-1,"20$3");

if ($lowdate > $highdate) { die "$usage\n"; };

open (IFH, "<$infile") or die "Could not open input file $infile\n";
open (OFH, ">$outfile") or die "Could not open output file $outfile\n";

$intag = 0;
$inTHEtag = 0;
$tag = "";

while (read (IFH, $newchar, 1)){

 if ($newchar =~ m|\<|) { $brack = 1 }
 elsif ($newchar =~ m|\>|) { $brack = 2 }
 else { $brack = 0 };

 unless ($intag == 1) {

  unless ($brack == 1) {

   print OFH $newchar;

  } else {

   $intag = 1;
   $tag .= $newchar;

  };

 } else {

  $tag .= $newchar; 

  unless ($inTHEtag == 1) {
  
   if ($brack == 2) {
  
    unless ($tag =~ m|^\$|) {
    
     print OFH "$tag";
     $tag = "";
     $intag = 0;
    
    } else {

     $inTHEtag = 1;

    };
   };
  
  } else {

   if ($brack == 2) {

    if ($tag =~ m|\$|) {
     
     if ($tag =~ m|\(\d{4})(\d{2})(\d{2})|) {

      $transdate = timelocal(0,0,0,$3,${2}-1,$1);     
      #print OFH "$tag " . scalar(localtime($transdate)) . " XXX";

     };

     $lowdif = $transdate - $lowdate;
     $highdif = $highdate - $transdate;


     if (
      (($transdate >= $lowdate) && 
      ($transdate <= $highdate)) ||
      ($transdate < ($lowdate - $lowtol)) ||
      ($transdate > ($highdate + $hightol))
     ) {
      print OFH $tag;
     };

     $tag = "";
     $intag = 0;
     $inTHEtag = 0;

    };
   
   }; 
   

  };
   
 }; 

};


close IFH;
close OFH;

HOWTO: Keep your passwords on a USB drive. Access them from different platforms.

I have a lot of online accounts and passwords, and it's becoming impossible to remember them all. So, I have been looking for an open-source, password-management tool to keep track of them.

The problem is that I use multiple computers, and multiple operating systems (Windows, Linux) on a regular basis, so ideally I need something that ...

(a) can be run from a USB device, and
(b) has a front-end GUI for multiple platforms / operating systems.

It appears that this can be done with Password Gorilla (http://fpx.de/fp/Software/Gorilla/).

Password Gorilla is a GUI application that stores passwords in a separate psafe3 database file that is encrypted with the Twofish algorithm.

Password Gorrilla provides an executable file for Windows. For a number of other operating systems, such as Linux, Mac, BSD, and Solaris, Password Gorilla comes in the form of a platform-independent "Starkit" file that will run once you have downloaded the platform-appropriate "Tclkit" from http://www.equi4.com/tclkit/download.html.

Instructions:

These instructions assume you already have a USB drive with a file system that can be read by Windows, Linux, or whatever other OS you're using. I used fat32, but you might be able to use ntfs, as well.

0. Create a folder/directory on the USB device. I called mine "gorilla".

1. Go to http://fpx.de/fp/Software/Gorilla/#Download and download both the Windows executable file, as well as the Linux/Solaris/*BSD Starkit file. Put them in the folder you created.

2. Go to http://www.equi4.com/tclkit/download.html and download the Tclkit for each of the other platforms you will be using. If you don't see the platform you want, look here for a larger list. Use version 8.4 or newer. I downloaded the Linux x86 32-bit Tclkit, version 8.5.1. Put the Tclkit(s) in the folder you created.

3. Make the Tclkit file(s) executable.

In Windows, you launch Password Gorilla by double-clicking (or running) the .exe file.

In Linux (and other *nix), you launch it as follows:

$ ./tclkit-linux-x86 ./gorilla-1.4.kit

I created an executible file containing that above line so that I wouldn't have to type all that every time and so that I could launch it in Linux using a file manager.

Here's what I have, now, in the "gorilla" directory on my USB drive (as seen :

$ pwd
/media/disk/stick/gorilla
$ ls -al
total 4288
drwx------ 2 gideon root 65536 2009-01-08 14:43 .
drwx------ 7 gideon root 65536 2009-01-08 14:46 ..
-rwx------ 1 gideon root 632 2009-01-08 14:15 gkdb.dat
-rwx------ 1 gideon root 1626485 2009-01-08 14:03 gorilla-1.4.exe
-rwx------ 1 gideon root 251221 2009-01-08 13:40 gorilla-1.4.kit
-rwx------ 1 gideon root 37 2009-01-08 13:58 linux-start
-rwx------ 1 gideon root 2180434 2009-01-08 13:40 tclkit-linux-x86
$ cat linux-start
./tclkit-linux-x86 ./gorilla-1.4.kit
$

The result is that I now have a single, encrypted, password database that lives on my USB drive, that can be accessed from both Windows and Linux. (I'll probably add support for Mac, eventually.)

Hiatus

Apologies for the hiatus. My cat died. I was traveling. I have been working on a real cluster for work. Progress to resume, soon.

VirtualBox Sun Cluster - Update 1

I haven't actually set up a working Sun Cluster yet using VirtualBox, but I have the component parts set up and working correctly.

Basically, a cluster needs at least two nodes that are individually able to run the application or service that is being clustered. These two systems must share at least two private network connections, be connected to the same public network, and (usually) have access to the same storage.

My goal is to set up a basic, fail-over cluster on my laptop, using two virtual machines as my nodes, so that I can practice basic cluster administration. I'm using Ubuntu Linux as my underlying, host operating system, VirtualBox (Sun xVM) for the virualization software, Solaris 10 for each of the virtual machine "guests", and iSCSI (served up by the host OS) for the shared storage.

First, let's look at my laptop:

Dell Inspiron 1525
Pentium Dual Core (not sure how fast ... maybe 1.5 GHz?)
2 GB RAM
160 GB Harddrive

This laptop is probably barely powerful enough to pull this off. I wouldn't want to try this with anything less that 2 GB of RAM. (I've tried it on a 1 GB laptop, and my single guest OS kept freezing.)

The underlying host OS will be Ubuntu 8.04 LTS Linux. I have the harddrive split up into multiple partitions, and it's set up to boot both Vista and Ubuntu via grub. I would have liked to set up a Solaris partition, since most of my expertise are in Solaris, but Vista and Solaris 10 did not play well together. I can't use Vista as a host OS, (a) because I know virtually nothing about it, (b) because I don't want to risk screwing up my ability to play WoW, and (c) because I'm not aware of free iSCSI target software for Windows, whereas iSCSI Enterprise Target is available for Linux.

More to come ....

VirtualBox Veritas Cluster

I've put the Sun Cluster project on hold for a bit in order to investigate the possibility of building a Veritas Cluster. I was able to get ahold of some trial license keys for VCS on Solaris, but the installer won't enable the software on my virtual nodes with them. My suspicion is that Solaris 10 x86 32-bit isn't supported by VCS, and isn't recognized by the installer / licensing software.

The keys work on a SPARC box, but I don't have a 64-bit x86 box to test this on, unfortunately, so I'll have to wait for a friend to help with this one.

VirtualBox Sun Cluster

My current project is to build a working Sun Cluster on my laptop using VirtualBox.

Basic setup:

Hardware:

• Dell Inspiron 1525 w/ 2GB of RAM

Host:

• Ubuntu 8.04
• JWM GUI (instead of Gnome, to conserve memory)
  
• VirtualBox 2.02
• Enterprise iSCSI target
  

Guests:

• Solaris 10 5/08 x86
• Sun Cluster 3.2 (w/ secret, unsupported 32-bit binaries)
  

I have found the blog and forum postings of other people very helpful:

Configuring the network:

• http://ubuntuforums.org/showthread.php?t=78293

Setting up iSCSI and Sun Cluster in VirtualBox:

• http://system-log.tyr.org.uk/2008/06/27/building-a-solaris-cluster-express-cluster-in-a-virtualbox-on-opensolaris/
• http://blogs.sun.com/mramcha/entry/solaris_cluster_on_a_laptop

Stay tuned for further details.

Inaugural Post

Did I spell "Inaugural" right? No idea.

The purpose of this blog is to provide me with a place to record the progress and outcome of the little technical projects I do in my spare time, in case anyone else is interested. When I can't figure out a particular technical problem, a Google search usually reveals that someone has already solved it. I'd like to return the favor.