Paul Weinstein

So weird, Connecting HavenCo and Red Hat

2012-04-01T03:30:28Z

It's a bit weird to be reading about Red Hat posting $1 billion in revenue in a year for the first time or this Ars article by James Grimmelmann about HavenCo since, to me personally that's part of my past.

See, as Grimmelmann notes, HavenCo's chairman of the board was Sameer Parekh whom I worked with/for at a different internet security company, C2Net Software. Almost everything Grimmelmann writes about I remember first-hand. I even remember reading the Wired articles he references (and how could I forget Neil Stephenson's Cryptonomicon, it's still one of my favorite novels).

Around the same time, Steven Levy wrote the non-fiction book Crypto, which tells part of the history of securing communications and modern computing networks; from Whitfield Diffie and the initial concerns of privacy to Netscape and the creation of SSL.

Alas, Levy's book is already 10 years old. While it covers the basis for the cryptography that powers today's Internet, it doesn't necessarily tell the whole story. Parts of the story that are missing, such as the short comings of SSL and its open standard successor, TLS, the adoption of "virtual private networks", that allow the use of primarily public networks, such as the Internet, to connect remote points securely, as if part of a central private network or that much of today's emails remain in "plaintext", despite the availability of encryption methods such as PGP, is missing.

Most of what happens on today's Internet every moment, took root around the same time of Levy's work, 1999-2001, when I was right there working for C2Net with its own vision on how to secure everyday communications on the "Information Superhighway".

And what happened to C2Net? Well it was sold, to......Red Hat of which I become an employee of (and then ex-employee of).

So yeah, I have this odd, I remember that (HavenCo) and oh, good for them (Red Hat). Then I think wow, I wasn't just a part of the some pioneering companies "back in the day", but also witnessed some completely cutting edge stuff that's only now being understood by the world at large.

So weird.

Chicago Open Data at Work

2012-01-23T14:59:26Z

A few years ago Blagica Bottigliero started the website Gals' Guide as an online forum for young women moving out on their own an into the "big city". Recently, I've been working with her on taking the site to the next step; building a web application utilizing the growing sets of data about life in Chicago.[1]

The Gals' Guide Map App is designed to combine different datasets about the city's various neighborhoods into one, assisting one in finding right place to live.

The web app is, somewhere between alpha and beta stages, not ready for general use or even rigorously browser tested, but ready for feedback. To that end, we've started showing the app to our various networks to gather feedback as it moves towards a general, full public release.

The map and features therein have been influenced by other map mashups out there, such as the recent work done by the Chicago Tribune's News Applications team.[2]

Currently it incorporates data from the U.S. Census, the City of Chicago and Groupon. But, that's just the tip of the iceberg. There are plenty of other datasets about the city from sources such as county and state, Everyblock, Yelp, Grubhub and others.

Go, check it out and leave some feedback.

[1] This is also the next logical step for me from coding up PHP classes for the CTA's API and the City of Chicago's open data portal I started working on back in July.

[2] The team has a blog which includes a nice series of post on their work, I recommend taking a look.

The Power of mod_proxy

2011-11-28T16:39:53Z

The Power of mod_proxy: An Introduction to Proxy Servers and Load Balancers with Apache HTTP Server
Presentations slides from my mod_proxy presentation at ApacheCon NA 2011 earlier this month. In addition to this slideshow, the presentation can be downloaded in PPT and MP3

Adding SQL Server Support in PHP on Linux

2011-10-31T23:00:36Z

Back in July I outlined a method for establishing a SSH tunnel between Linux and Windows machines. The goal of the connection was to enable a PHP script on a front-end Linux web server access to information stored on the back-end private Windows server running SQL Server.

What I didn't mention at the time was how I enabled PHP support for Microsoft's SQL Server.

The most common deployments of PHP on Linux include support for MySQL or Postgres, depending largely on other factors such has the organization's preference, experience and requirements. Since PHP can be deployed on Windows, there is support for Microsoft's SQL Server. Such support is nontrivial to enable in PHP on Linux. It is however possible:

To enabled SQL Server support in PHP on Linux, the PHP extension that provides said support requires the FreeTDS library to build against. FreeTDS is an open source implementation of C libraries originally marketed by Sybase and Microsoft to enable access to their database servers.

Downloading the source code, building and installing FreeTDS is straightforward:

$ wget \
ftp://ftp.ibiblio.org/pub/Linux/ALPHA/freetds/stable/freetds-stable.tgz
$ gunzip freetds-stable.tgz
$ tar xf freetds-stable
$ cd freetds
$ ./configure
$ make
$ make install

The next step is to build the PHP source code against the FreeTDS libraries to include SQL Server support. This can be done one of two ways; build PHP from scratch or build the specific PHP extension. Since I was working on a server with a preexisting install of PHP, I opted for door number two:

Locate or download the source code for the preexisting version of PHP. Next, copy the mssql extension source code from the PHP source code into a separate php_mssql directory:

$ cp ext/mssql/config.m4 ~/src/php_mssql
$ cp ext/mssql/php_mssql.c ~/src/php_mssql
$ cp ext/mssql/php_mssql.h ~/src/php_mssql

Now build the source code, pointing it to where FreeTDS has been installed:

$ phpize
$ ./configure --with-mssql=/usr/local/freetds
$ make

There should now be a mssql.so file in ~/src/php_mssql/modules/ that can be copied into the existing PHP install. Once copied the last remaining steps are to enable the extension by modify the php.ini file and restarting the Apache HTTP Server.

Additional Information can be found here: Connecting PHP on Linux to MSQL on Windows

Accessing Chicago, Cook and Illinois Open Data via PHP

2011-09-30T20:38:07Z

In Accessing the CTA's API with PHP I outlined a class file I created¹ for querying the Chicago Transit Authority's three web-based application programming interfaces (APIs). However, that isn't the only open data development I've been working on recently, I've also been working on a class file for accessing the City of Chicago's Open Data Portal.

The City of Chicago's Open Data Portal is driven by a web application developed by Socrata. Socrata's platform provides a number of web-based API methods for retrieving published datasets². class.windy.php provides the definition for a PHP object that wraps around Socrata's API providing PHP written apps access in turn to the city's open data.

Installation and Instantiation
The first step is to download the class.windy.php file from GitHub and save it in a location that the PHP application can access.

The next step is to include the file using the include function in the PHP application itself:

	// Load our class file and create our object.
	include_once( 'class.windy.php' );

Once the class file has been loaded, the next step is to instantiate the class:

	// Create an object to get city data
	$chicago = new windy( 'city' );

Note that initialization of of the new object requires providing the keyword 'city'. Since the City of Chicago, Cook County and the State of Illinois all currently use Socrata's platform for sharing government data, class.windy.php supports accessing all three data portals - more on accessing county and state data later.

Extra parameters can be provided during the construction of the object for additional customization. For example, by default, this class will request data in JSON form. But, Socrata does support other data formats³ so the PHP developer can have a bit of a choice as to how to handle and process the raw data, if desired.

Chicago Data
Let's say for example that one is looking for information about Chicago's roughly 200 neighborhoods. More specificity information about the rough boundaries around Chicago neighborhoods. Using the getViews⁴ method that class.windy.php provides, one can query for any dataset where the description might include the keywords 'neighborhood boundaries' and are tagged as KML:

	// Let's find any views that describe themselves as about 
	// Chicago's neighborhood boundaries and are tagged with 
	// KML data
	$views = $chicago->getViews( 
		'', '', 
		'neighborhood boundaries', 'kml', 
		'', 'false', '', '' 
	);
	
	echo "Here are the views with a description including 
	     'neighborhood boundaries' and are tagged as KML:\n";
	foreach ( $views as $view ) {
	
		echo "View ID: ".$view->id. " is named " 
		     .$view->name. " and is described as a " 
		     .$view->description. "\n\n";
		
	}

Depending on what datasets are available the output of this query may include the follow result:

View ID: buma-fjbv is named Boundaries - Neighborhoods - KML and is described as a KML file of neighborhood boundaries in Chicago. To view or use these files, special GIS software, such as Google Earth, is required.

Perhaps, using Google Maps, one wishes to create an interactive map of Chicago neighborhoods? Or better yet a closer look at the boundaries of a specific area in the city? Retrieval of the KML file can be done via the getFileByViewID function:

		// With our foreknowledge of datasets the file with 
		// view id buma-fjbv looks interesting, let's get 
		// that file
		if ( $view->id == 'buma-fjbv' ) {
					
			$file = $chicago->getFileByViewID( 
				$view->blobId, $view->id 
			);

			// Since KML is an XML notation for 
			// expressing geographic annotation, 
			// let's use SimpleXML to parse this data
			// Note: SimpleXML requires the libxml 
			// PHP extension
			$xml = simplexml_load_string( $file );

The above chunk of code nested in the previous example code's foreach loop compared the $view->id value of each dataset looking for a match to the specific KML file of Chicago neighborhoods. Once found, the getFileByViewID method, given the file id, found in $view->blobID and the known view id sends a query for and retrieval of the file from data portal.

Once the file has been loaded, the KML file, a type of XML format, can be parsed using the simplexml_load_string function which returns a SimpleXML object and can be traversed as needed:

			// Ok, now let's find out what the 
			// boundaries are for Albany Park
			foreach ( 
			     $xml->Document->Folder->Placemark 
				as $hood 
			) {
			
				if ( preg_match( 
					'/Albany Park/', 
					$hood->description 
				)) {
				
					echo "Here are 
						Albany Park's 
						boundaries: " 
						.$hood->
						MultiGeometry->
						Polygon->
						outerBoundaryIs->
						LinearRing->
						coordinates. "\n";
				
				}
			
			}

Thus the output of our search for the boundaries of Chicago's Albany Park neighborhood results in the the following geographic data points:

Here are Albany Park's boundaries:  
-87.724211,41.975689,0.000000
-87.724081,41.975576,0.000000 -87.723951,41.975578,0.000000
-87.723773,41.975580,0.000000 -87.723544,41.975583,0.000000
-87.723355,41.975585,0.000000 -87.722786,41.975592,0.000000
-87.721267,41.975605,0.000000 -87.720888,41.975608,0.000000
-87.720859,41.974088,0.000000 -87.720825,41.974085,0.000000
-87.720636,41.974088,0.000000 -87.720561,41.974079,0.000000
-87.720501,41.974061,0.000000 -87.720370,41.974010,0.000000
-87.720264,41.973981,0.000000 -87.720135,41.973962,0.000000
-87.719949,41.973890,0.000000 -87.719723,41.973819,0.000000
-87.719616,41.973775,0.000000 -87.719252,41.973625,0.000000
-87.719147,41.973578,0.000000 -87.719074,41.973535,0.000000
-87.718948,41.973461,0.000000 -87.718826,41.973396,0.000000
-87.718714,41.973346,0.000000 -87.718388,41.973226,0.000000
-87.718366,41.973218,0.000000 -87.718291,41.973195,0.000000
-87.718069,41.973097,0.000000 -87.717935,41.973023,0.000000
-87.717737,41.972915,0.000000 -87.717479,41.972781,0.000000
-87.717195,41.972637,0.000000 -87.717132,41.972604,0.000000
-87.716961,41.972517,0.000000 -87.716548,41.972280,0.000000
-87.716203,41.972086,0.000000 -87.715963,41.971978,0.000000
-87.715918,41.971968,0.000000 -87.715808,41.971941,0.000000
-87.715666,41.971921,0.000000 -87.715501,41.971920,0.000000
-87.715076,41.971932,0.000000 -87.714697,41.971930,0.000000
-87.714681,41.971930,0.000000 -87.714619,41.971930,0.000000
-87.714614,41.971930,0.000000 -87.714547,41.971937,0.000000
-87.714479,41.971942,0.000000 -87.714435,41.971949,0.000000
-87.714385,41.971952,0.000000 -87.714305,41.971954,0.000000
-87.714227,41.971960,0.000000 -87.714178,41.971965,0.000000
-87.714026,41.971969,0.000000 -87.713652,41.972033,0.000000
-87.713466,41.972078,0.000000 -87.713254,41.972129,0.000000
-87.712834,41.972277,0.000000  ...

Cook and Illinois Data Portals

As previously mentioned the Cook County and the State of Illinois also currently use Socrata's platform for sharing government data. For the moment that makes supporting city, county and state open data with one class file a trivial manner:

// Create an object to get county data
$cook = new windy( 'county' );

// Let's find any views that describe the boundaries of the 
// county forest preserves
$views = $cook->getViews( 
	'', '', '', 'county park boundaries', '', 'false', '', '' 
);
	
echo "Here are the views with a description including 
	'county park boundaries':\n";
foreach ( $views as $view ) {
	
	echo "View ID: ".$view->id. " is named " 
		.$view->name. " and is described as a " 
		.$view->description. "\n\n";
	
}

In Review
class.widy.php is a single PHP class file that primarily provides access to City of Chicago's data portal. The class implements functions for accessing all API methods and by default returns an object that a PHP developer can use to incorporate information about the City of Chicago into their PHP based application.

Since Cook County and the State of Illinois have also adopted Socrata's data platform, support for these additional data portals using Socrata's Open Data Platform has been included as a secondary feature of the class file.

1 And which can be found on GtHub

2 Known as SODA, a standards-based, RESTful application programming interface.

3 Such as XML, RDF, XLS and XLSX (Execl), CSV, TXT and PDF.

4 getViews is one of a collection of methods Socrata refers to as ViewsService. This collection of API calls provide for the retrieval and manipulation of datasets and metadata about datasets.

Accessing the CTA's API with PHP

2011-08-22T14:00:34Z

Overview
Last month the City of Chicago arranged for a Open Data Hackaton in which a collection of programmers gathered together to develop and write programs that utilize a new resource, open access to city information.

For my part, I spent the data writing a PHP class file that wraps around the Chicago Transit Authority's web-based application programming interface, enabling access to CTA bus, rail and service information for PHP driven applications. As I've noted in the README file, "this class brings all three APIs together into one object with related methods."

The following in a quick rundown of how to incorporate this new class file into a working PHP application.

Installation
The first step is to download the class.cta.php file from GitHub and save it in a location that the PHP application has read access.

The next step is to include the file using the include (or similar require) function in the PHP application itself:

// Load the class file in our current directory
include_once( 'class.cta.php' );

Once the class file has been loaded, the next step is to instantiate the class:

$transit = new CTA ( 
	'YOUR-TRAIN-API_KEY_HERE', 
	'YOUR-BUS-API-KEY-HERE', false 
);

Notice that initialization of transit includes providing two API keys. API Keys can be requested from the CTA. For an API Key for Train Tracker, use the Train Tracker API Application form. For Bus Tracker, first sign into Bus Tracker, then request an Developer Key under "My Account".¹

If no valid API keys are provided the only methods that will return valid information are the Customer Alert functions for system status information. Specificity the two functions statusRoutes and statusAlerts. This is because the Customer Alert API does not require an API key for access.

Execution
To invoke a method simply use the object and related function, providing any additional information as parameters, if required. For example, to get information about all of the bus stops the east-bound route 81 bus makes:

// Get an array result of all stops for an east-bound 81 bus.
$EastBoundStops = $transit->busGetStops( '81', 'East Bound' ));

All methods return an array which can be accessed to retrieve desired information. PHP's print_r or var_dump functions provide insight into all information returned by a specific function:

echo '';
print_r( $transit->busGetStops( '81', 'East Bound' ));
echo '';

The output will look something akin to this:

SimpleXMLElement Object
	(
	    [stop] => Array
	        (
	            [0] => SimpleXMLElement Object
	                (
	                    [stpid] => 3751
	                    [stpnm] => 2900 W Lawrence
	                    [lat] => 41.968500785328
	                    [lon] => -87.701137661934
	                )
...
	            [49] => SimpleXMLElement Object
	                (
 	                   [stpid] => 3725
	                    [stpnm] => Milwaukee & Higgins
	                    [lat] => 41.969027266773
	                    [lon] => -87.761798501015
	                )

 	       )

	)

In order to generate the following output listing the location of the Lawrence & Kimball stop:

Lawrence & Kimball (Brown Line)
At 41.968405060961 North and -87.713229060173 West

The following PHP code will provide the latitude and longitude of the Kimball stop, which is also a transfer point to the El's Brown Line:

$EastBoundStops = $transit->busGetStops( '81', 'East Bound' );
foreach( $EastBoundStops as $stop ) {

     if ( preg_match( '/kimball/i', $stop->stpnm )) {
		
          echo $stop->stpnm;
          echo 'At ' .$stop->lat. 'North and ' .$stop->lon. ' West';
		
     }
	
}

Notice that while the list of stops is provided in an array, each element in the array is a SimpleXMLElement object, thus the use of the object syntax for accessing each element.

The train function will allow for the determination of rail information, for example when the next Brown line train will be leaving the Kimball stop. However, while the previous example included a stop id for the route 81 bus at Kimball, the stop id is unique to the route 81 bus and does not translate to the stop id of the Brown line El at Kimball. Therefore, the first step is to locate the relevant GTFS² data for the Kimball station:

/*	Per the CTA's website, El routes are identified as follows:

	Red = Red Line
	Blue = Blue Line
	Brn = Brown Line
	G = Green Line
	Org = Orange Line
	P = Purple Line
	Pink = Pink Line
	Y = Yellow Line
		
	Which means our Brown line is 'brn' 
*/
$brownStops = $transit->train( '', '', '', 'brn' );
foreach( $brownStops as $stop ) {

	if ( preg_match( '/kimball/i', $stop->staNm )) {
		
		echo "$stop->staNm train is destined for $stop->stpDe ";
		echo "Scheduled to arrive/depart at $stop->arrT";
		
	}
	
}

Which provides output similar to the following:

Kimball train is destined for Service toward Loop

Scheduled to arrive/depart at 20110821 17:17:01

One should note that the Brown line stop at Kimball is the northern end point for the Brown line, which means any trains leaving the station will only be bound in one direction, south, toward the Loop. If the string comparison is changed to 'irving' for the Irving Park station, the output changes to something similar, with trains running in both directions:

Irving Park train is destined for Service toward Kimball

Scheduled to arrive/depart at 20110821 17:19:34

Irving Park train is destined for Service toward Kimball

Scheduled to arrive/depart at 20110821 17:23:13

Irving Park train is destined for Service toward Loop

Scheduled to arrive/depart at 20110821 17:19:44

Irving Park train is destined for Service toward Kimball

Scheduled to arrive/depart at 20110821 17:32:19

In Review
class.cta.php is a single PHP class file that provides access to all three CTA APIs for Bus, Train and Service information. The class implement functions for access to all API methods and returns an array of SimpleXMLElement objects that a PHP developer can use to incorporate real-time information about Chicago's public transit system.

Additional information about the CTA's APIs, including terms of use and how to request API Keys, can be found on the CTA's Developer page.

1 Why two different API Keys, one for train and one for bus information? Due to the evolution of the CTA's API interfaces, there are three distinct APIs, one for Bus, Train and Customer Alerts information. As a result there are three distinct URI endpoints and two distinct API keys.

2 The CTA provides its data based on the Google Transit Feed Specification (GTFS) which is becoming a common format for public transportation agencies to publishing schedules and associated geographic information. The CTA generates and distributes, about once a week, an up-to-date zip compressed collection of files that includes basic agency information, route, transfer and stop locations, and other related service information. Note that ids 0-29999 are bus stops, ids 30000-39999 are train stops and 40000-49999 train stations (parent stops).

Chicago's Technology Scene, circa 2011

2011-08-02T18:00:00Z

Roughly this time two years ago, I attended an entrepreneur and startup event in D.C. called Social Matchbox which I noted in a subsequent blog post of the same name.

While I know organizations and companies like those exists in many places I have yet to find a loose confederation of those individuals, organizations and companies similar to what I experienced in the Bay Area here in Chicago where I currently reside. I have however found such a network in Washington, DC and it is known as Social Matchbox.

Last year, I asked with all the things going for Chicago, high profile tech companies, top tier universities and a diverse population, "why don't tech people think of Chicago along the lines of a San Francisco, Seattle, New York, Boston or Austin?" Perhaps, I surmised, it was simply "that Midwestern work ethic, if we just work hard; the rewards and recognition will come on their own."

This year, I watched as the Chicago tech community rallied around TechWeek, a "celebration of a new technology epicenter unique among major world cities."

It's a start.

Alas, while I didn't get to attend any TechWeek specific events, someone has to keep an eye on the servers and write code for all these newfangled ideas, I did get a chance to meet up with a few other developers at the Chicago Open Data Hackathon. As a WTTW article wrote:

Chicago's city government has worked on developing its high-tech cred by initiatives such as publishing new city data sets online weekly and refreshing those sets nightly in order to increase the city's transparency ... On July 16, Google Chicago hosted a Chicago Open Data Hack Day. The event gathered 60 engineers, designers, and entrepreneurs to share ideas about using the City's open data to create new products and services."¹

In my own participation of the hackathon, I did get a chance to create, what I hope will be, a useful PHP library (more on that later).

It's good to see Chicago get a little more boisterous about its tech creds. But it is also good to see Chicago go about about business as most Chicagoans do, as I did last week. As Orbit's Andy Crestodina notes "in a refreshing way, TechWeek was nothing new. Chicago has been doing this a long time and the tech community is an experienced crowd, many of whom have seen the boom and bust (and more booms and more busts) and lived to tell about it."

There is an end in all this technology means. We build apps to communicate. We open data to map relationships and piece out new meaning. The technology isn't an end unto itself. We work in technology to get something done.

And that's the innovation Chicago can bring to the table, even if it mostly goes unheralded.

1 I don't recall 60 people offhand, I probably would have put the count at around 30-40. But the hackathon was an all-day event and I know some people came and left, so, 60 total for the whole maybe true.

Establish and Maintain an SSH Tunnel between Linux and Windows

2011-07-06T14:00:16Z

The Situation
Over the years, I've worked in numerous computing environments and have come to appreciate heterogeneous systems. In my mind, all system administrators should experience how different platforms solve similar problems, just as all programmers should be exposed to different programming languages.

Of course this means being able to play well with others. Sometimes, that's easier said than done.

A recent project requirement stipulated being able to connect a public web server with a private database system. Not an uncommon requirement, but it did place a hurdle immediately in the way. The web application, developed with the Linux, Apache, MySQL and PHP (LAMP) stack, needed a method to connect to the private database system securely, which, for fun was not MySQL but instead Microsoft's SQL Server.

The Problem
The initial requirement called on connecting to the SQL Server using Microsoft's virtual private network (VPN) solution, Microsoft Point-to-Point Encryption (MPPE). Not impossible, since support for MPPE on any Linux distribution simply requires modifying the Linux kernel and recompiling the kernel in Linux is usually a non-issue.

However, in this case the web application would be running on a basic virtual private server (VPS) and a Linux VPS doesn't run its own kernel. Instead Linux VPSes run on a shared kernel used by all the different virtualized servers running on the same hardware.

Net result, no modification of the Linux kernel would be possible on the VPS.

One alternative to this hurdle would have been to switch from a Linux VPS to a Windows VPS. This would have been technically possible since Apache, MySQL and PHP have viable Windows ports. Alas, the hosting provider in question didn't yet offer Windows VPSes. They would shortly, but couldn't guarantee that their Windows VPS solution would be available in time for this particular project's deadline.

A second alternative could have been to upgrade from a virtualized server to a dedicated server. But that would have added more computing resources than what was required. From a business perspective, the added monthly cost wasn't justifiable. Not when a third alternative existed.

A Workable Solution
VPN is one of those terms that can refer to something generic as well as something very specific¹. This distinction setups up alternative number three. The secure network connection requirement would remain, the implementation could simply change².

Specifically the secure connection would be implemented via SSH instead of via MPPE.

With SSH an encrypted tunnel through an open port in the private network's firewall can be established. This tunnel forwards network traffic from a specified local port to a port on the remote machine, securely.

Most Linux distributions these days install OpenSSH as part of their base system install. OpenSSH is a free and open version of the SSH protocol and includes client and server software. For those distributions that don't install it by default installing OpenSSH is usually a trivial matter via the distribution's package manager.

Windows, on the other-hand, has no such base installation of an SSH implementation. There are a number of free software versions for Windows. For the case at hand, freeSSHD was selected to provide a free, open source version of the SSH server software.

Configuring freeSSHD to enable tunneling requires the following steps:

Click on the "Tunneling" tab
Check to enable port forwarding and apply the change
Click on the "Users" tab
Create or edit a user and enable tunnel access

Once the firewall has been configured to allow SSH traffic on port 22, establishing the tunnel from the Linux client to the Windows server is as simple as typing the following at the Linux command-line:

ssh -f -N -L 127.0.0.1:1433:192.168.1.2:1433 username@example.org

In which ssh will create and send to the background a ssh tunnel (-f option) without executing any remote commands (-N option) that begins at the localhost port 1433 (127.0.0.1:1433) terminates at the remote address and port (192.168.1.2:1433) and authenticates using the remote username at the remote location (the public IP address or domain name for the private network).

But Wait There's More
There is however a minor problem with this SSH tunnel. As described, the establishment of the SSH tunnel is an interactive process. The command needs to be executed and the password for the user provided for authentication. In most cases a simple shell script, executed by cron would solve this minor issue. However, for the sake of security OpenSSH doesn't provide a command-line option for providing passwords.

This authentication step can be managed in one of two ways. One is the use of a key management program such as ssh-agent. The second, more common option is to create a passphrase-less key.

The first step in creating a passphrase-less key is to first generate a private/public key pair>sup>3. In Linux this is done by issuing the command:

ssh-keygen -t rsa

Which generates a private/public key pair based on either the RSA or DSA encryption algorithm, depending on what is provided in the command-line option.

When prompted to enter a passphrase for the securing of the private key simply press enter. To confirm the empty passphrase simply press enter again.

The next step, after copying the public key onto the Windows server, is to enable the use of the public key for authentication. In freeSSHD the steps are:

Click on the "Users" tab
Select a user and click on "Change"
Select "Public Key" from the "Authorization" drop-down
Click on "OK" to save changes to users
Next click on the "Authentication" tab
Using the browse button, select the directory with the users public key are kept
Enable public-key authentication by choosing the "Allowed" button under "Public-Key Authentication"
Click on "OK" to save the changes to authentication

With the passphrase-less keys in place, the last step is to automate the tunnel itself. In this case, instead of a shell script, I opted to use program called autossh.

autossh is a program that can start a copy of ssh and monitor the connection, restarting it when necessary. All autossh needs to know is what local port to monitor, so our one-time initial startup of ssh tunnel looks similar to the previous example, but with autossh and the addition of the -M option

autossh -M 1433 -f -N -L 127.0.0.1:1433:192.168.1.2:1433 
username@example.org

[1] This means alas, it is also one of those terms that can cause confusion, especially between technical and non-technical people, if not defined at the outset.

[2] This is one of those places where knowledge of different solutions solving a similar problem becomes handy.

[3] For user authentication SSH can either be password-based or key-based. In key-based authentication, SSH uses public-key cryptography where the public key is distributed to identify the owner of the matching private key. The passphase is in this case is used to authenticate access to the private key.

Speaking at ApacheCon NA 2011

2011-06-13T13:30:53Z

At the moment November, and another hockey season, seems a long way off. But I'm already making plans to be in Vancouver from the 9^th to the 11^th for the North American edition of ApacheCon.

That is because, at present, I am schedule to present at ApacheCon on The Power of the mod_proxy Modules:

This presentation reviews the concepts of web proxies and load balancing, covers the creation and maintenance of proxies (forward and reverse) for HTTP, HTTPS and FTP using Apache and mod_proxy and how mod_proxy_balancer can be used to provide a basic load balancing solution. Configuration Examples of implementing proxies and load balancer will be discussed including; how and when mod_proxy modules can help, configuring mod_proxy for forward or reverse proxy and configuring mod_proxy_balancer for one or more backend web servers.

A preview of this upcoming presentation can be found in a previous talk from ApacheCon EU 2006, entitled Apache 2.2 and mod_proxy_balancer.

For those who might be interested, this year's ApacheCon is set to cover:

Enterprise Solutions
Cloud Computing
Emerging Technologies & Innovation
Community Leadership
Data Handling, Search & Analytics
Pervasive Computing
Servers, Infrastructure & Tools

More information, including full schedule and registration, can be found at: http://na11.apachecon.com/

16 of 20: Personal Artifacts from the Early Years of Linux

2011-05-30T17:45:00Z

As the story goes, in 1991 Linus Torvalds, then a student at the University of Helsinki, began work on the core component, the kernel, of his own operating system based off of Andrew S. Tanenbaum's teaching operating system MINIX.

Since releasing the source code 20 years ago, and under his guidance ever since, the Linux kernel has become central to the development of devices from smartphones to super computers.

But for most people, when they think of Linux, they think of Linux distributions, software that includes the Linux kernel and supporting resources that complete the basic requirements of an operating system.

A few weeks ago the Linux.com Editorial Staff posted an article on one of the earliest distributions of Linux, Slackware. That in turn got me thinking about some of my early exposure to Linux in the late 90s, and what media I might still have from that time.

My personal copy of Andrew S. Tanenbaum's Modern Operating Systems from my Computer Science days along with Daniel A. Tauber's The Complete Linux Kit and Randy Hootman's Linux - Installation and Beyond

A quick scan of my bookshelf revealed a couple of interesting artifacts. One is a copy of a book, "The Complete Linux Kit", a 1995 title that included a CD-ROM with the Slackware Linux distribution¹ that was compiled by Daniel A. Tauber and printed by Sybex.

The second is a video, "Linux - Installation and Beyond" which is described as a "three hour seminar showing installation of Red Hat Linux, Slackware Linux, Yggdrasil Plug-and-Play Linux" featuring Randy Hootman and was distributed by Yggdrasil Computing. Alas while Red Hat and Slackware still exist, in one form or another, the Yggdrasil distribution is no longer maintained and the company no longer exists.

In a small effort to share what Slackware was like, "back in the day" I offer this copy of Randy Hootman's tutorial on installing and using Slackware Linux version 2.3:

As a small bonus, and because I personally worked for Red Hat, once upon a time, here is Randy on Red Hat Linux v2.0:

Speaking of Red Hat, a screenshot of my Linux Workstation in 2000, running GNOME v1

[1] Where the CD has since gone to, I'm note sure . Probably in a book of CD-ROMs in the storage.

The Beginning of the End of the Ballmer Era at Microsoft?

2011-05-16T13:30:00Z

Something has been puzzling me for awhile now. I've been wondering, where is the outrage, the angst and the out right hostility toward Microsoft CEO Steve Ballmer? I mean where are the industry pundits, Wall Street analysts, shareholders and Microsoft insiders calling for his head?

And then I saw Ben Brooks' blog post this past week entitled The Ballmer Days are Over and I realized, here it is, the beginning of the end of Steve Ballmer at Microsoft.

What am I babbling about?
Microsoft is Microsoft because of exactly two pieces of software, Windows and Office. Without Windows and Office there would be no Microsoft. No Xbox, no Windows Phone. Nor would they have been able to buy their way online with Hotmail or Skype.

But Microsoft has a problem. The market for its crown jewels have matured. The PC industry has stagnated. Sure, Microsoft can release a new version of Window or Office every few years, but customers, individuals and businesses, no longer feel compelled to spend several hundred dollars on the very latest version of either product.

Worst yet, sentiment seems to have turned away from Microsoft, if I'm replacing an old PC with a new one, the consumer thinking seems to go, why not just go ahead and get a Mac?¹

So, what is Microsoft's future, if not Windows and Office upgrades?

The Internet Tidal Wave
Since May 1995, Bill Gates' company has set its sight on a future dominating the online world in a manner similar to the desktop. But from AOL to Facebook, someone always seems to beat them to the punch.²

That in itself probably explains their $8.5 billon acquisition of Skype, beat Google or Facebook for once.

But beat them to what exactly? As Andy Ihnatko pointed out on his own blog, there are three reasons why Tech Company A might buy Tech Company B. Note that none of them are "beat Tech Company C to the punch."

The closest justifiable reason from Ihnatko's list is "Company A gets to 'level up' instantly without having to spend years engaged in a long, expensive and uncertain dungeon crawl."

Yet, it is not like Skype is the hot new it online and as Brooks hints at, how many mistakes would Microsoft have to make to make $8.5 billon a reasonable trade off to "level up"? Surely nobody thinks "Apple spent anything close to $1 billion dollars building FaceTime?"

Sure Microsoft gets a technology it can instantly drop into Windows and Office their by given users a reason to upgrade. But, if they had developed something on their own, dropped that technology into Windows and Office for free, they could have tens of millions of potential paying customers of their own online conferencing service instantly for potential a lot less money.³

Yes, of course the business unit that makes money off of Windows and Office upgrades loses money on a free upgrade, but remember Windows and Office are the past, online is the future.

Well, maybe online is the future? Here's the rub, and why I've been thinking Ballmer's days are numbered for awhile now, Microsoft doesn't know what exactly their future online is. Don't believe me? Take a look at this:

Microsoft is chasing everybody online and gaining on nobody, they a literally losing millions of dollars each quarter. Add in the setbacks Microsoft has had in other business areas; mobile, with Zune and Kin and the challenges ahead for the next generation of gaming consoles against their one true success, the Xbox and well, one has to wonder, how much longer does Steve Ballmer have as CEO of Microsoft?

[1] Apple loves to tell people that since the opening of their stores 10 years ago, the majority of customers walking out with a new computer are first time Mac owners.

[2] Think about it, Internet Explorer the answer to Netscape, MSN and Hotmail was Microsoft's answer to AOL, Bing to Google, et al.

[3] Of course, we can't forget the legal trouble Microsoft got itself into the last time it tried to integrate online features into the core of Windows.

Saying No to Transparency Budget Cuts

2011-04-13T12:36:54Z

In the past few years we have seen an explosion in our ability to access information at anytime and from just about anywhere. The Internet in general has a lot to do with this explosion. Being able to access the Internet from just about anywhere has simply reinforced its importance.

A secondary development that has impacted our relationship to information is known as "Web 2.0". The central tenant of Web 2.0 is that websites not only facilitate in the sharing of information, but also the interoperability of information across the Internet.

To do this web developers separate out their concerns, the design of the website from its logical behavior, the information from the logic. This allows greater flexibility in developing the over all web-based application, applying computing resources specific to the concern in question, say storage of application data, as needed.

Given a proper interface a web developer can create a website "mashup" pulling data from numerous "outside" resources into a unique and useful application. One of earliest examples of this type of web application was a website called chicagocrime.org [1] that combined Google Maps with the Chicago Police "blotter" to provide a digital map in which a user could locate informtion about criminal activity at a given location in the city.

Recently the Obama Administration launched a number of government transparency initiatives designed to create data stores of federal information, akin to the local police blotter. By providing these data stores the administration's goal was to increase public access to high value information in a format that could be easily incorporated into a larger web application.

Yet, some of the most important technology programs that keep these data stores available are in danger of being eliminated. Data.gov, USASpending.gov, the IT Dashboard and other federal data transparency and government accountability programs are facing a massive budget cut, from $34 million to $8 million or less.

Government information must be available online, in real time and in machine-readable formats. Doing so can increase involvement in our democracy as non-for-profit organizations, for-profit businesses and independent developers find new ways to enable us in accessing and sharing information.

It seems stupid to let these new initiatives go dark. After all, why should we be able to know what is going on in our neighborhood, but not in our country as a whole?

Take action now to Save the Data, http://sunlightfoundation.com/savethedata

[1] Chicagocrimes.org has since morphed into EveryBlock.com, a site focused on collecting all of the news and civic goings-on related to a specific city neighborhoods, As of March 2010 they cover 16 metropolitan cities in America and their neighborhoods, providing a "news feed" for a given city neighborhood or block.

Let's Play Two (or Web Analytics for Fun and Profit)

2011-03-31T22:51:44Z

Back in October it occurred to me that it had been 5 years since the White Sox beat the Astros to win the World Series. As I result of that realization, I dug into my video collection and quickly put together and posted this video:

To say that this video is the most popular video I've posted on YouTube thus far is an understatement. What's more interesting, for those of us who work the medium of the web, is the traffic statistics of those who have viewed the video in the past 5 months:

So what do these stats tell us? Well to some extent it tells us a few things we might have already "known", such as that most baseball fans (or at least White Sox fans) are mature males residing in the United States.

What I find interesting is when people were viewing this little video. Obviously some people viewed it right when I posted it last Oct, during the 2010 World Series. Then, as expected, things go quite for the most part. Then, as Spring Training builds to today's Opening Day, so does the traffic.

But wait, you might be wondering, what about the spike of traffic in December?[1] What could possibly have driven the largest one-time surge in traffic for a handful of days? Perhaps I engaged in a little social marketing? Or maybe the video got popular on a sports site?

Well as it happened it did get posted on a local sports site, but that doesn't completely explain the surge, or why said site was posting a baseball video in December.

Why did it get popular so quickly (and fade so quickly) in December? Because, on December 8, 2010, Paul Konerko, the hero of the video, resigned with the Chicago White Sox for 3 more years.

Interesting, No?

[1] Well if you really are a White Sox fan, you might not be wondering, but don' spoil the ending please?

Remembering Challenger

2011-01-28T16:39:14Z

I was walking down the hallway. It was lunchtime and I was heading toward the gymnasium. The hallway was empty, always a minor thrill of independence for a grade-schooler. No classmates, no teachers and certainly no parents.

I don't recall exactly where I was coming from, but I hadn't yet passed my classroom. Then my teacher, Mrs. Y approached. I recall her telling me something had happened and knowing my interests, suggested I join Mr. H's class. He had originally gathered his class together to watch Christa McAuliffe and her crewmates launched into Earth orbit.

The desks in Mr. H's room faced to the left, toward the large chalkboard. But the class was sitting on the floor, facing a television strapped into a tall cart straight ahead of the door. For the first time, I saw the footage of what had happened prior to my arrival.

Astronaut Memorial at Kennedy Space Center by pdweinstein via Flickr

We naturally filter what occurs around us personally. Being in 5^thgrade at the time I can only say I knew of the crew that made up Challenger's last attempt at orbit. Yet looking back 25 years later, I can't help but think I was personally affected by their loss, for a childhood dream was slipping away.

I remember two and half years later my classmates and I reconvened to witness the return to flight of the Space Shuttle. But two and half years is a life-time and a half to a preteen. What might have been a personal interest to many was expressed as distinct uninterest in just another mandatory group lesson.

Middle school is the beginning of that hazy inbtween of childhood fantasy and adult responsibility. One knows the expectation is to "think realistically about one's future". No longer is it acceptable to answer the eternal question of "what do you want to be when you grow up" with the response of astronaut. Yet what else is there?

I still carry a personal interest in space and space exploration with me. I follow the politics and as well as the technological developments. Perhaps some day I might still be able have a small influence, in something that has had a large influence on me.

Where Software Methodology and Business Strategy Meet?

2011-01-10T17:15:03Z

Image by pdweinstein via Flickr

When I initially envisioned the article Web Development: Before and After the Client (local copy), my initial idea was to draw a line connecting Orbit Media Studio's overall business strategy, through the implementation of specific development methodologies, to the end result of the type of service Orbit offers.

Consider the description for Orbit's method of enhancing their content management system; "the focus is on breaking down the feature into workable steps and rapidly building them." This incremental process allows Orbit to keep coding solutions simple, quickly incorporate lessons learned from previous projects and sustain development of their codebase for the long term;

In doing so we consider what has worked for clients in the past along with growing trends such as social media integration.

But once a client enters the picture the goals change. A client has specific objectives, chief among them the desire for a stable website that is delivered on time and on budget. The change in goals requires a change in methodology;

In this sequential development process each step follows from the last. There is a specific beginning and ending. One step cannot be started until the previous step is completed and approved.

Thus, while these two methods have distinctly different goals unto themselves, Orbit uses both to bring about a specific end result in desired quality of results.

The point that I was trying to make got me wondering. Given a set of goals would most developers, or project managers, choose the development method(s) best suited to realizing those goals? Or would they choose the one they are most comfortable with?

Sure, most project managers or software engineers would recognize the different methodologies described above and that they focus on different objectives. But what's the most common way a development process is chosen?

Unfortunately, my experience over the years leads me to believe that most choose the one they know the best, is in fashion or is what is used "in-house." In fact many places I've interviewed at consider development methodologies like software platforms, they maybe an "Agile shop" just as much as they are a Mac or PHP shop.

This lack of connection between desired outcome and actual process haunts both business and technology managers in my opinion.

Then again, maybe I've had an odd experience with my career thus far?

Postscript: In perhaps an answer to my own question, while I was drafting this blog post, I came across this article, Design Driven Development. The post, from a Boston-based web design firm, outlines a similar organic evolution of their development process.