The Burden of files

Okay, there’s a lot of problems:

• Backing up your files is a pain in the butt.
• Every time you upgrade your system, either in-place or a fresh install, it is a royal hassle to restore your files.
• That file looks great on your laptop, but how do you show it to a colleague not sitting next to you easily? Ughh.
• You upload files to a random directory on some web server you have some space on, quickly to show an idea to someone. Fast-forward some time, and you’ve got disorganized, poorly-named files scattered across multiple shell / other accounts all over the web, and you’re not sure what you have a copy of where, or which ones are being referred to from other places, so you’re terrified to delete any of them.
• Well, crap. You’ve made a mistake. You can’t go back, can you? No version control…

I think we all know these problems pretty well. I’ve built a solution using Fedora and Sparkleshare – completely free and open source software – that over the past week has addressed all of these issues and has substantially improved the quality of my computing life. It backs my work files up to an internal corporate server and it backs my Fedora files up to a Fedora-maintained public server. I’m planning to configure it to back up some personal files to my Dreamhost account and some to my NAS at home.

What? What are you talking about?

Here’s how it works: I have a ‘Sparkleshare’ folder in my home directory. Under that, I have a couple of subfolders:

• workstuff
• design-team (this is for Fedora stuff)

When I start a project, say it’s for work, then I create a new subfolder under the appropriate directory. So I’m starting a new project, let’s say it’s a logo design for project X. I have the following directory tree:

~/SparkleShare/workstuff/logos/

I open up Inkscape, and start working on the logo. I work on it for a while. Now I’ve got a draft I’d like to save and show to a colleague. From within Inkscape, I navigate to the SparkleShare > workstuff > logos folder and save the logo there. Then I can go to my internal server, and automagically, the logo file I just saved is at http://hostname/workstuff/logos/mylogo.svg. I can quickly copy/paste that URL from my browser and send it via IRC or email to my colleague.

No uploading, no fumbling to find a server that has space in an appropriate location, no waiting for the upload to happen, no coming up with some lame variation of ‘temp’, ‘tmp’, ‘foobar’, ‘blah’ to make a fresh folder to upload it to. Nope. It’s just there.

Here’s how to set this up for yourself.

Setup Walkthrough

Ingredients

You will need:

• A desktop or laptop computer with Fedora 14 or Fedora 15 installed. You’ll be needing root access.
• A remote server you’re going to use as your main git repository. This can be a server you administer yourself, or it can be a shell account with lots of space on a hosting provider like Dreamhost. It could even be a NAS if you can install git on it like you can on mine.
• An ssh key, preferably with a passphrase. I am assuming you already have your SSH key configured to enable you access to the remote server you’re intending to set your main git repo on. If not, the following resources below might be helpful:
  • What is an SSH key and why do I need one? (Documentation from the Fedora project.)
  • How to create an SSH key (Documentation from the Fedora project.)
• Files. Lovely little (or big) files.

Other assumptions:

• I’m assuming you’re using GNOME. Sparkleshare does work on KDE or XFCE or LXDE. What you lose by not using GNOME is the nautilus right-click extension that lets you – from gitorious or github repos – check out older versions of a document or grab a web link to the document. Not a huge deal.

Step 1: Prepare your local system

We’re going to install Sparkleshare first.

1. Open up a terminal.
2. Go to the yum repositories directory:

   cd /etc/yum.repos.d

3. As root, download Alex Hudson’s Sparkleshare Fedora people repo config:

    sudo curl -O http://repos.fedorapeople.org/repos/alexh/sparkleshare/fedora-sparkleshare.repo

4. Install sparkleshare and openssh-askpass:

   sudo yum install -y sparkleshare openssh-askpass

5. Remove nautilus-python (I don’t know why, but if you have this installed and install Sparkleshare, Nautilus will segfault on startup. If you remove it, though, everything works fine.)

   sudo yum remove -y nautilus-python

You now have Sparkleshare installed.

Step 2: Set up your main git repo

First you have to figure out where you want your main git repo to be. It needs to be in an account that your desktop/laptop can connect to. I set mine up in a remote home directory I own. Here’s how to do it:

1. We’re going to create the empty new git repo. In your remote home directory, type

   git init --bare repo.git

   . (If this command doesn’t seem to work and you have permissions to do so, try installing openssh-server and updating git.)

2. Now we’re going to make a dummy clone of the repo to add an initial file and branch. To make things more interesting, we’re going to do this in your public_html directory, but you can do it anywhere you’d like in your home directory:
   • Go into your public_html directory.

     cd ~/public_html

   • Clone it! Type

     git clone repo.git

   • Go into your new clone with

     cd repo

   • Now make a new dummy file,

     echo 'Fedora 15 rocks!' >> test.txt

   • Now commit it.

     git add test.txt; git commit test.txt -m 'initial commit2'

   • Push it, making your initial branch.

     git push origin master

Step 3: Hook your local system up to your repo

Okay, now we’re going to get your local system hooked up to the repo you just created on your remote system.
On your local laptop/desktop:

1. Turn Sparkleshare on. Click on its icon in your Applications menu, or run it from the terminal:

   sparkleshare start

2. Right-click on the Sparkleshare icon in your system tray, and select “Add Remote Folder…”
3. Where Sparkleshare asks you: “Where is your remote folder?” select “On my own server.” In the field after “On my own server, replace my fake credentials here with your own, and don’t forget the ‘.git’ on the end.

   user@hellokitty.ponies.com

4. Where Sparkleshare asks you for the “Folder name”, fill in something like this, replacing my directory structure with your own on the remote server:

   /home/user/repo.git

5. Sparkleshare should say something about ‘coffee o’clock’ and it should start pulling the remote files down.
6. Look in your Sparkleshare folder for the repo you just connected. Are the files from the server there?
7. Now we’re going to make sure it’s set up right. Create a text file in your new Sparkleshare repo directory (will be somewhere like ‘/home/user/Sparkleshare/repo’).

Okay, that should have pushed up to the server, no issues. Make sure it did; On your remote system where git is:

1. Go into your remote clone.

   cd ~/public_html/repo

2. Pull down the changes!

   git pull

3. Make sure the changes worked. You can either visit the web side of your clone at (replace with the relevant hostname and username) http://hellokitty.ponies.com/~user/repo or you can just cat it, making sure our new panda message is present:

   cat test.txt

All right, hopefully that wasn’t an issue either and your remote clone was able to pick up the panda PSA you pushed from your local laptop/desktop. Now we’re going to throw Sparkleshare into the mix.
On your local laptop/desktop where Sparkleshare is installed:

1. Restart Sparkleshare. You can click on the icon or

   sparkleshare start

2. Go to the Sparkleshare repo directory.

   cd ~/SparkleShare/repo

3. Make a new test file. In honor of towel day today:

   echo '42' >> theanswer.txt

Let’s see if it worked!!!!! On your remote system:

1. Go into the clone:

   cd ~/public_html/repo

2. Pull it down!

   git pull

Did it work??? It worked for me!!

Step 4: Set up automagical web mirroring of your repo’s content

Okay, so I did a sneaky thing. Our remote repo, in the examples above, has been configured in your remote public_html directory on the same system your remote git repo is. Now, you could follow these directions on a third remote system if you’d like; no problem. Just make sure you’ve got your public SSH key configured on that third system and that you have access to write to a web-readable directory. These instructions should not be difficult to modify for that case; simply git clone your repo in a web-readable directory.

We are now going to hook this all up so that when you save a file on your local laptop/desktop, and SparkleShare checks it in, the web-readable clone of it all also automagically pulls down your change and makes the file available in a web browser. Woo!

On your remote system:

1. Get into the hooks directory where your main (not clone) git repo is (don’t forget the .git!):

   cd ~/git.repo/hooks

2. We’re gonna set up a hook so whenever you push a file to the repo, it tells the web clone to pull. Create a file called ‘post-receive’ in ~/git.repo/hooks with the following in it, replacing ‘user’ with your own username on the remote system:

   #!/bin/sh
   
   while read oldrev newrev refname
   do
   true
   done
   cd /home/user/public_html/repo
   GIT_DIR=/home/user/public_html/repo/.git git pull
   

Okay, let’s see if it worked.
On your local laptop/desktop where Sparkleshare is installed:

1. Make sure Sparkleshare is running. If it’s not, start it.

   sparkleshare start

2. Go into the right directory and make another test file.

   cd ~/SparkleShare/repo; echo 'Fedora 15' >> latestfedora.txt

Okay, moment of truth. Is your new test file in your web directory? Either visit (replacing the username and hostname of course) http://hellokitty.ponies.com/~user/repo and look for a file named latestfedora.txt (if it’s there, it worked!) or on the remote system:

1. Go into the web-readable repo clone:

   cd ~/public_html/repo

2. Look for the file.

   ls

It worked!!

What do you think?

So, what do you think? Pretty awesome, right? Goodbye, dropbox – a worthless solution when your corporate IT policy rightly frowns upon copying internal documents to third-party public servers. You can create as many repos like this as you like. You can host them on a remote server you have shell access to, as this tutorial assumed, or you can set it up on Gitorious (I recommend this one since it’s backed by free & open source sw) or Github. Or a Fedora Hosted repo if you’ve got one. Each repo will have a folder under your ‘SparkleShare’ folder in your home directory, and anything in that directory is basically remotely backed up, instantly, as soon as you save files in those directories (and given that Sparkleshare is running.)

We’re currently experimenting with using this setup for the Fedora Design team to hopefully make it easier for folks to access our files and contribute.

Some things you might want to consider with this setup

• Set Sparkleshare to always run when your machine is booted. For GNOMEies:

  cp /usr/share/applications/sparkleshare.desktop ~/.config/autostart

• Write down the remote git URLs of all of the remote repos your stuff is backed up to and keep it in a safe place. If disaster strikes and a panda mistakes your laptop for bamboo, it will make it that much easier to recover.

Major kudos

• Hylke Bons and the rest of the Sparkleshare community for making such an awesome, life-changing tool.
• Alex Hudson for creating and maintaining the Fedora people repos Sparkleshare repository.
• Ray Strode, Kevin Fenzi, and Ricky Zhou for fielding my crazy git questions. Ray wins the prize for figuring out how to get past git complaining about a ‘sideband demultiplexer’.

Emily has put together some beautiful use case-centered tri-fold brochures she’s going to blog about for you soon. I spent some time earlier this week putting together the lovelies below which will be available as handouts at our expo booth, #344, starting on Monday.

We were really excited to hear from Chrissie Brodigan of Mozilla today, who is giving a talk tomorrow at 2 PM, “How to Not Design Like a Developer.” She’ll also be beating the drum for getting more designers involved in free & open source software. Go Chrissie!

Free & Open Source Assets

This flyer reviews some of the best sites out there to get freely-licensed assets, such as Open Clip Art Library, Open Font Library, CCMixter, and Blend Swap, among other great sites.

This flyer also talks a bit about asset licensing, pointing out that just because something is free that doesn’t mean you can use it commercially, modify it, or redistribute it. It talks about how Fedora requires all three of these things to accept content, and goes over some of the Fedora-approved content licenses.

We’d love to get more awareness for these awesome collaborative sites and hopefully inspire some SXSW-goers to join in and collaborate in creating high-quality content for these sites as well.

What Can Free & Open Source Software Do?

This flyer reviews some great works done by creatives using free & open source creative tools, including the Blender Institute’s films (Sintel, Big Buck Bunny, Elephant’s Dream).

With this flyer, we’d like to show that free & open source tools really are capable of producing amazing, professional-quality, and beautiful works.

Who Uses Fedora?

This flyer is a reprint of some of the interviews we did of actual Fedora users for fedoraproject.org‘s recent redesign. We wanted to put a face to some of the creatives already using Fedora, talking about how they use specific free & open source creative tools that are available in it to produce their works.

Guess Who Uses Free & Open Source Software?

In this flyer, we call out some of the big creative and interactive companies and brands that use open source. In researching this flyer, I was impressed to see how many projects Sony Pictures ImageWorks have open sourced, including what appears to be a color management system. They also use Cinepaint! They are one of the examples on the flyer.

How to Get Involved with Open Source: a Creative’s Guide

Here’s five tips for creative-minded folks who’ve thought about helping out a free & open source software project but weren’t sure where to start. A lot of free & open source communities are still quite centered around developers and other technical folk, so this was our attempt at writing something specific for creatives and addressing the particular challenges and cultural issues they might face in trying to get involved.

Even if you can’t be at SXSW, I hope these materials are interesting and useful, and show how awesome free & open source tools and culture really are. If you’d like to remix them, the Inkscape sources are available.

These flyers are available under a Creative Commons Attribution 3.0 license except for particular elements which are called out in the footer of each individual sheet.

What can we do to help expand the reach of free software? One way is to teach! I’ve taught free software to kids and it was an amazing experience (and I have another Red Hat-sponsored project involving teaching kids how to use free software in the works that I’m excited to talk about in detail soon!) Although interacting with a group of kids or adults in person and teaching them about what free software can do is vitally important, I am afraid it’s not entirely scalable. It’s also entirely bending the user to the software itself, without at all bending the software towards the user. “Yeah, it works a little weird there, here’s a clunky workflow for getting around that” is the sort of phrase I end up using a lot of times while teaching others how to use free software.

I am concerned that bending users to the software, and not bending the software to the users at all really limits the immense impact free software and our community can have on the world. I fear that teaching and evangelizing is not enough to expand the availability of software freedom to folks who need it; we need to improve our offerings to require a bit less of a technical wall to jump in order to enjoy them.

This proposition understandably has made some folks – experienced users who are technically-savvy and long-time free software users – nervous. Free software is a great thing. Is working towards making it more mainstream going to ruin it, just like that awesome indie band you used to love that made it big and now totally sucks? I share the same fear, and yes that’s happened to music artists I loved too…. but I think for us there is little risk, and the rewards – sharing the gifts of software freedom with more people – are totally worth it.

Here’s why I think the proposition to focus more on new Linux users is not that risky for current users:

• If the default setup of a given Linux OS changes, it won’t necessarily prevent you from doing what you need or want to do. If we focus the default experience to cater towards new & casual users, it really should not make a huge difference to power users – it should not exclude them – I think at worst it will necessitate some work-arounds and initial customizations for the power users (but I think many of you are already doing these things today.)
• Making improvements to benefit less-familiar users can benefit everyone, and at worse simply not effect experienced users either way. The way I like to think about this – for me, tidying up the mess in the living room is not really fun and it’s liveable as-is, so there’s little motivation to do it all that often. When we tidy things up for guests coming over, though, we benefit from the organized living room just as much as our guests do! The improvements should help everyone, and not exclude anybody.

Let me provide you with a bit more thinking on each of these points:

1: Changed defaults won’t exclude power users

So here is a little case study, using desktop configuration as an example since the desktop tends to be a visually-rich experience that can be offered up for comparisons relatively easily: I asked around for Fedora users’ screenshots in IRC this past week – actual, working screenshots, not artificially-polished & cleansed screenshots. This is how a slice of Fedora users actually use Fedora, folks. We’ve got GNOME Shell, we’ve got heavily-customized KDE, we’ve got XFCE, even icewm. Panels are moved around, different icon themes and widget themes are installed, folks are using multiple different mail and chat clients in these screenshots:

View a slideshow of all these screenshots

Looking at these, can you understand why someone would argue as vehemently against a new-user friendly change in say the desktop default settings? Do you think a change of the defaults is really going to affect any of these users at all? None of these folks mentioned or seemed to feel excluded by the current desktop defaults, and many readily admitted they love to tweak and customize. It seems many current Fedora power users are quite capable and even happy to customize and change how they use Fedora well beyond the default setup.

2: Why helping new users won’t hurt you, and why ignoring them hurts us

A big part of the reason a lot of us are here in the Linux community is to improve & spread software freedom and free culture. Fedora’s mission in particular is, “to lead the advancement of free and open source software and content as a collaborative community.”

There’s a couple of things you need to realize that goal:

• You need contributors. You need contributors to create and help improve free software, to create and license compelling content under free licenses, to help make your project run.
• You need users. Sure, it’s fun to have the secret navel-gazing treehouse where we proselytize to the already-converted, but that’s not really effective if you want to actually spread your message outside of your circle.

How do you get contributors? You recruit from your pool of users! How do you get users, to widen your potential contributor pool and to spread your free software / free culture message? You reach out to them, providing them a compelling reason to care. Okay, great, that’s easy right? We just get out there and send our message out – it’s a great cause – folks will want to help, right?

Well, no. I believe that trying to convince new folks to get involved based solely on free software / free culture dogma is not going to work. Even if you get them convinced to try to help, you’re going to put them into this dilemma:

Not cool. It’s like you’re getting kids under the drinking age all fired up about a new club, and when they actually show up, they are bounced at the door. How rude! If you’re going to recruit folks like this to help Linux out, Linux needs to be something they can be inspired by – something they can actually use. Otherwise, why will they care? And for the few who either are inspired already and see the potential, or who find out about free software & culture on their own and have some interest in it, it’s not just that they have to gear up just to be able to join your project – there’s alternatives calling out to them that are more welcoming and far easier to get started with:

How often have you seen this problem in your project? Isn’t this something that could be solved and benefit everyone if we could only attract more contributors?

Lowering the bar to enter ‘the ride’ that is Linux is going to mean that more people can use and contribute to it. Lowering the bar won’t exclude folks who are already ‘tall enough’ to get into ‘the ride’ today.