I'm building yet another new toy in PHP (and no, I haven't finished the others...) and I ran into a familiar problem. I use ICDSoft for my hosting for appsCanadian, since when I bought the hosting I only intended to host this blog on it and I didn't need anything fancy or expensive. I just went with the cheapest option that had positive reviews. Now, I'm not knocking ICDSoft at all. In fact, they've been a perfectly good host, and they've been amazingly responsive to any and all inquiries I've had. However, being an economy host, I'm working on a shared server with no ssh access. This means that I have no ability to install Subversion, which is still my go-to version control system, largely since I've never bothered to use Git or Mercurial.

This presents a minor problem for me, since I really like developing on this server, for various reasons like the fact that my laptop is running Windows and I'm just paranoid enough to not want to develop on a different OS than I'll ultimately be deploying to. My reasons aren't that important, and if you're reading this it's likely that you have your own reasons for wanting to develop on some remote host you can't install things on or ssh into.

I've tried to find workarounds for this, and last time around I decided that I would just run Ubuntu in a virtual machine on my laptop. I found this to be a bit of a burden as I don't normally spend a lot of time using Linux, although I am relatively comfortable working with such a box, and I didn't like having to boot up a whole other machine just to make a quick change to my project. This time around, I decided to find a better way.

After a bit of brainstorming and a few searches on Google, I stumbled across WinSCP which is an open source (S)FTP/SCP client for Windows. The main feature that hooked me was the scripting features, which have commands for synchronization and directory monitoring. I've written a small script which will allow monitoring:

sync.txt
option batch on
option confirm off
open WINSCP_SESSION
option exclude .svn/
synchronize remote -delete "LOCAL_DIRECTORY" "REMOTE_DIRECTORY"
keepuptodate -delete "LOCAL_DIRECTORY" "REMOTE_DIRECTORY"

And I run pass this script to WinSCP from a Windows batch file:

sync.bat
"C:\Program Files\WinSCP\WinSCP.com" /command /script=sync.txt

The execution of this script is pretty straightforward. First we open a particular WinSCP session, defined in advance via the GUI, which contains credential information for your (S)FTP/SCP connection. Next, we do an initial synchronization of the remote directory with the contents of the local directory, opting also to delete any remote files which no longer exist locally. You can also replace the "remote" option with "local" to synchronize the other way around, or use "both" to do a two way sync moving the most recent files from each side to the other (note that the delete option obviously has no effect when synching both ways). Finally we use the keepuptodate command to tell WinSCP to monitor your local directory and push any changes up to your remote directory. Since the whole point of this exercise is to be able to develop remotely while still being able to checkout a working copy from svn locally, I've also used an exclusion option to tell WinSCP not to propogate any changes having to do with a .svn directory, since this is Subversions work area and it's never needed on the server.

With this script running, I can make any and all changes necessary to my project on the local files, and as I save these changes WinSCP will quietly send them to my server where I can see them immediately. I just have to be careful not to make any changes directly to the server, or I'll run the risk of having them overwritten by any local changes that get made.

One final caveat. There seems to be a known issue where a directory deletion made locally will NOT get propagated to the remote directory, though it seems its contents will be deleted without issue. It seems that because WinSCP has the directory open to monitor changes within it, Windows isn't able to complete the deletion until the monitoring process ends. Because the deletion isn't completed until the monitoring process ends, the completed deletion won't be noticed by WinSCP and will therefore not be applied to the server. It's a bit of a catch-22. I did make a bug report on the WinSCP forums, but the developer responded saying he's seen the issue before but is unsure how he can fix it. I'm therefore not going to hold my breath for a bug fix, but luckily there's an easy workaround. If you either need to recreate the directory locally (Windows won't allow its recreation until the deletion process of the original can be deleted, for obvious reason) or if the existence of the directory on the server is a problem, you can simply restart the batch script and the initial synchronization process will clean up for you.

Since WinSCP is open source, I would love to be able to contribute a fix to this, but the code is in C++ and a little lot above my ability. If anyone out there knows how to fix this issue, I'm sure a patch would be appreciated by the developer.

I hope that this information can help someone else who finds themselves in a similar situation to mine. If so, I'd love to hear from you if for no other reason than to convince myself that I'm not crazy for using such a setup.

I read a blog post just now via Hacker News where a guy called Matt Swanson was lamenting that someone else had beaten him to the punch and launched a website similar to an idea he had. By the end of the post, he had come to the conclusion that this wasn't the end of the world - it taught him that the idea had merit and he was simply going to compete with them, and that he could still win because he was building a better mousetrap.

This post sort of struck a chord with me, since I had the same experience a number of months ago. In fact, my experience was almost identical since my idea was also a tech-oriented book site. Finding these competitors is definitely a blow, but I'm risen above it by taking stock of my motivations and realizing that the product was never the point for me.

After a few years of working with ancient technologies such as COBOL, I found in a bit of a depression since my ambitions had been corrupted. I felt rather like the character of Marshall Eriksen on the CBS sitcom How I Met Your Mother. Marshall began his legal career with the goal of becoming an Environmental Lawyer, but found himself working for an evil megacorporation to pay the bills. He's in the right industry, but he's no longer pursuing the career he wanted.

My goals for my book site are a lot different than Matt's: I'm not so much developing a product so much as trying to teach myself something new. This will be the first website I've ever developed from scratch, and it's taking a long time because I'm being a bit of a perfectionist with the project. I figure if this is worth learning, then it's worth learning right. It's much harder to unlearn something you've learned incorrectly than it is to learn it right the first time. Learning something right, however, is a more time intensive process.

So Hacker News didn't ruin my morning like it ruined Matt's. My goals haven't changed, and my goals were never based on being the first mover for this idea. My goal was to use this project to develop my skills so that next time around I'll be better equipped to compete. If I learn well this time make the development process second nature to me, I can compete on execution alone next time. Maybe then I'll win.

Laying Things Out On The Web

October 19th, 2010

We've already talked about the basics of separating the logic of your application from its presentation, and we've put in place the building blocks of how your code will load its components as needed. Before we talk about creating the Controller classes which will handle incoming requests and determine their output, I want to talk a bit about creating the HTML templates that will make up that output.

Fixed vs. Fluid Layouts

There are two main methods to laying out a website - fixed layouts and fluid ones. There are also methods which combine these two, but they're beyond the scope of this discussion.

In a fixed layout, the site is structured not to exceed a certain width. These sites tend to be contained within a single <div> container element, which has been set to be exactly a certain pixel width. From there, the site can be designed largely as if it were on a piece of paper. If you have an image that you know is 200x200 pixels, and you know that your site will always be shown with a width of 1000 pixels, you know that your picture will take up one-fifth of the horizontal space on the page. Because of this ability to create pixel-perfect designs up front, fixed width designs are much easier to create than fluid designs.

Fluid designs, on the other hand, have no concept of a hard site width. Instead, element widths are specified as percentages of the total horizontal screen real estate available. As a result, content can easily stretch to use all available horizontal space in the users browser, eliminating dead, unused space.

To better visualize the difference between these two layouts, I have created two very simple pages to illustrate each layout:

Sizing Your Wrapper

Largely because I'm new to web design in general, I'm going to use a fixed-width layout for my site. I figure it's best to start with the easy stuff before moving on to more involved techniques.

The first thing I need to do is decide what size I want my site to be. The two leading sizes are 760 pixels and 960 pixels, which roughly correspond to the 800x600 and 1024x768 screen resolutions, respectively. These numbers were decided by taking the horizontal width of the screen (800 and 1024 pixels, respectively) and shaving off a bit to accommodate things like scroll bars and other browser chrome.

Since 800x600 resolutions are pretty rare these days, most people opt to design using the larger 960 pixel option. Those with lower resolutions will see a horizontal scroll bar at the bottom of their screen when using your site. A good tip would be to use the right-most 20% of your design for less important, or non-essential information, as it is this portion of your design that would be "hidden" by default for users with an 800x600 resolution. 20% of a 960 pixel design is 192 pixels, roughly the difference between the 760 and 960 pixel options.

Creating the Wrapper

Creating the wrapper for your fixed-width layout is dead simple: just place a <div> around your content, styled with the following CSS:

#wrapper {
  width:  960px;
  margin: 0 auto;
}

This CSS breaks down as follows:

  • The #wrapper identifier just says that the following styles only apply to the element called 'wrapper'. There are lots of ways we could have specified this, like using a class rather than an id, but since we're planning on wrapping everything inside this element, it makes sense to use the id since there will only be one of them.

  • The width property has already been discussed at length, so we don't need to go over that again here.

  • The margin property is a bit interesting. The two values given are just shorthand, which says "remove margins from the top and bottom of this element, and automatically set its left and right margins." Setting the left and right margins to margin the element the same way on both sides, effectively centering the element on the screen.

One Container, or Many?

One thing I find a bit odd about a lot of the sites I've peeked at the markup for is that they all use a single massive container to hold all of their content. I'd like to propose that we can separate this massive container into smaller ones.

Instead of this:

#container {
  width:  960px;
  margin: 0 auto;
}

<div id="container">
  <!-- stuff... -->

</div>

We could instead have this:

.frame {
  width:  960px;
  margin: 0 auto;
}

<div id="header"  class="frame"> <!--header  stuff--> </div>
<div id="content" class="frame"> <!--content stuff--> </div>
<div id="footer"  class="frame"> <!--footer  stuff--> </div>

I think it gives a little added flexibility, and modularizes your page, at little extra cost. Just a thought.

Conclusion

Hopefully this has shed a bit of light on how different layouts work, and can get you started with building your first one.

Protect Your Source

September 19th, 2010

If you are serious about your programming project, then you should be serious about your source code too. Since your source code is your product, you need to protect it from getting damaged and becoming corrupt and unusable.

You might wonder how a digital file could get damaged, since in the general sense it doesn't have a physical form. I mean, if you take a picture on film, that film will naturally degrade over time, as will the pictures developed from it. Digital pictures, on the other hand, are just bits of information stored on a drive. If you make a copy of a copy of a copy of that original file, the picture contained in that copy will be exactly the same as the picture in the original. Since source code is safely locked away in the same sort of files, then surely it will stay in a pristine state in exactly the same sort of way, right?

Wrong!

Unlike your photos of your grandparents 60th anniversary, which are static items that are created and never changed (or which have some minor one-time touch-ups just after they're created to get rid of things like red eye), your source code is in a constant state of flux. As you test your software, you will inevitably find bugs in it, and those bugs will need to be fixed. When you do this, your code changes. When you find a better, faster way of performing a task, you refactor your code to take advantage of the new method, and your code changes. When you introduce new features, you may need to make tweaks to your existing product to make the new features work, and your code changes. Have you realized the issue yet? The biggest danger to your code is you!

Fortunately for you, this is a common issue. So common, in fact, that there already exist a variety of products to help save you from yourself. Collectively, these products are referred to as source control (or revision control, or version control, or configuration management, or...) and they allow you to keep a record of all the changes you make to your code and, if necessary, undo them. There are many different version control systems out there, but the one I'm going to use is called Subversion (often abbreviated 'svn').

With Subversion, all of your code is kept in something called a repository. The repository is a directory or database controlled by the Subversion software that keeps a running history of everything you have ever changed about a directory under Subversion's control. Each change to a repository is called a revision, and by giving Subversion a specific revision number, you can retrieve a copy of how your source code looked at any period in its history.

When you first create a repository, it is said to be at "Revision 0" which is a state where your repository holding just an empty directory. You create a new repository by telling the 'svnadmin' command to 'create' it at a specific location in your file system:

$svnadmin create ~/repositories/my_project

After you create the repository, you should consider the directory which contains it to be off limits. The only process that should interact with the repository at all is Subversion itself. When you have changes you would like to make, you will make them in a directory called a "working copy" which is a copy of the data held in the repository that you can use and modify without actually changing the canonical version in the repository (until you want to, anyway; more on that later). To create a working copy, simply create a directory to hold it anywhere you would like, then tell Subversion to "check out" a copy of the most recent revision of a specific repository:

$mkdir ~/my_project
$cd ~/my_project
$svn checkout file:///home/user/repositories/my_project

When you checkout a repository, Subversion will output a list of files and directories it creates (though in this example there are no files or directories in the repository to create), and it will output a message telling you which revision of your repository you're working with (e.g. "Checked out revision 0.").

In order for Subversion to properly keep track of the changes you make in your working copy, you need to babysit it a little bit. While Subversion will notice when you change a file already under its control, changes to the file system (for example, creating new files or directories, or moving them to new locations) have to be explicitly identified to Subversion. This is done by issuing certain file system altering commands from within the 'svn' application, i.e. using "svn cp", "svn rm", "svn mv" and "svn mkdir" rather than simply issuing "cp", "rm", "mv", or "mkdir" commands directly to the file system [I'm assuming a *nix OS here, but the Windows commands also work - "svn copy", "svn del", "svn rename"]. Note that when you create a brand new file in your working copy, this too needs to be identified to Subversion, using the "svn add" command.

Once you are done making modifications to your code in your working copy, such as after finishing writing a new function, you can tell Subversion to create a new revision in your repository which contains the updates now present in your working copy. To do this, you use the "svn commit" command:

$svn commit -m "Message explaining changes."

As you can see from this example, while you are able to specify a repository location when committing, it is not necessary. If no repository is listed, it is assumed that you are committing to the same repository you checked out from. The "-m" in the command is used to specify a "commit message," which is a note that you give to Subversion to outline the changes you made to the repository. This is useful for when you want to look back at previous revisions and quickly see what you did on each commit, possibly to find a specific change you made. It is also useful when you are working as part of a team of developers, rather than individually, because then the other developers can be quickly brought up to speed about the changes made to the repository since they checked out their copies by simply reading these summaries of what has been changed. When using the "-m" switch, you specify your commit message within quotes immediately thereafter. If you need to commit a longer message, you can write your message to a file before committing, then pass the filename to Subversion as part of your commit by using the "-f" switch. If neither the "-m" or "-f" switches are used, Subversion will check to see if you have defined a text editor program in a $SVN_EDITOR variable. If you have, Subversion will open the editor for you to enter your message when you try to commit. When you close the editor, Subversion will read your file to determine your commit message. If the $SVN_EDITOR variable isn't defined, however, Subversion will issue an error message and will abort the commit until a message is provided. If, for whatever reason, you decide not to include a commit message (not recommended!), you can simply pass an empty string to the "-m" switch during your commit.

Before committing your changes, however, there are two additional commands which you should get in the habit of utilizing:

The first command is "svn update" which checks if any updates have been made to your repository since you checked out your working copy. If there are changes, "svn update" will download them and update your working copy. If the updates interfere with your changes, Subversion will notify you of the conflict, and ask you to resolve them for it (either by choosing one copy or another to be used, or - more frequently - by updating your working copy to integrate the changes manually). You should try to "svn update" your working copy often while you make your changes, to reduce the number of conflicts you have to handle at one time, and to ensure you are always developing against up-to-date code. It is especially important to run this command immediately before attempting to commit your changes, as Subversion will not allow you to check in changes unless your working copy is up to date.

The second command is "svn status" which will give you a list of all changes you've made to your working copy since checking it out. The output of this command is simply a list of files that have been changed, preceded by a character indicating the type of change made. The four most common characters used for this purpose are as follows:

  • A - This indicates that a new file or directory is to be added to the repository for the first time.
  • D - This indicates that the file or directory is to be deleted. Note that this simply means that the file will no longer be listed with future revisions. Earlier revisions in the repository will still show the file.
  • M - This indicates that the file has been modified in the working copy since it was checked out.
  • ? - This indicates that Subversion has found a file or directory in your working copy that it isn't aware of. This happens largely when you create a new file in your working copy, but simply forget to use "svn add" to indicate to Subversion that you want to add it to the repository.

When you commit a change to Subversion, a new revision is created in its history. The real power of Subversion comes from the understanding that your new revision does not overwrite previous revisions - it simply replaces them as the most up-to-date copy of your repository. Every revision you have ever created still exists within Subversion, and can be accessed and reviewed. This allows you to "look back in time" and see earlier versions of your code, and allows you to understand why changes were made, or to undo those changes entirely.

Hopefully this has been a meaningful introduction to source control, and Subversion in particular. For more information, there is an excellent book called "Version Control with Subversion" which explains Subversion in amazing detail. You can download a PDF copy of this book for free at http://svnbook.red-bean.com or, if you prefer to read your books in dead tree format, you can order a physical copy from Amazon (check it out on Amazon.com or Amazon.ca).

The Project Begins

September 16th, 2010

Learning new programming languages isn't always the most fun experience if all you're doing is making trivial toy apps that whatever book you're reading decided was simple enough to fit their pages. The real learning, and the real fun, in programming comes from sinking your teeth into a real, non-trivial, useful application.

For a long time, I've struggled with coming up with project ideas. I really couldn't come up with much that excited me beyond the same simple crap used in those book examples. Fortunately for me, that problem seems to have disappeared. Over the last few months, I've come up with three separate projects that I want to build, and each new idea has felt more worthy than the last.

Now, I've decided that enough is enough, and it's time I got off my ass and built one of these projects. Actually, I decided this a few months ago, but unfortunately this decision came right when my wrist finally decided it had been abused enough. I've spent the past three months or so trying to work out my RSI symptoms (I'll likely write about that later), but I'm finally able to type without pain for reasonable stretches of time, so it's time to get to coding.

Since the project is probably pretty trivial for most web programmers to hack out, I'm going to keep the actual idea to myself for a while. In fact, a few days ago I discovered a new site that is a bit of a competitor, but I'm hoping that by aiming and designing for a specific niche of users will eventually allow me to win - or at least peacefully coexist - with this other site.

In the days and weeks to come, I'm going to use this space to write about whatever I learn while completing this project. Since I've never done a significant web project before (see the "toys" problem above), and my CSS is pretty rusty, I'm hoping that there will be enough good content to write about to justify my taking up your valuable time.

I'm also hoping this will become something of a two-way street, with your comments serving to correct my misunderstandings (and mark my words: there will be misunderstandings)and to help me become a better developer creating a better product.