July 4th, 2008
I have moved all my endeavors to http://jacob.steelsmith.org/ and have converted this WordPress blog into Drupal. I wasn’t able (or patient enough) to get the comments or categories posted over, but oh well! 
Just a quick post. To get the number of files in a directory under linux, use the following command:
$ ls -l | wc -l
This will output the number of files in the present working directory which can be output by using the command ‘pwd’.
Automatix is an automated installation program that offers many installations that are either unavailable in Ubuntu (for legal reasons or otherwise), or difficult. Automatix includes many installs such as Acrobat reader, Wine, Crossover, which allows the installation of Windows programs like Office and PhotoShop, pesky proprietary codecs, libcss2 for decrypting DVDs and much more. These are all installed using a GUI.
To install Automatix, follow the instructions here for your distribution. Once you install it, the menu item will appear under Applications-> System Tools.
If you have a long job using the command line in Linux, here are some tips.
Some jobs, such as compiling a program or using vacuumdb in postgres require a long time. Normally, you cannot use the same command line when running these jobs. If you are using ssh remotely, you’re stuck (unless you login again).
Now you can add a space and ampersand at the end of the command and this *should* send this to the background. But I’ve found this doesn’t always work. But even if it does, the following does still apply.
Once the command is started, you can press CTRL+Z to stop the command (not kill the command but pause it). You can then type bg to send the command to the background and resume the task.
If you type ps -a, you will see the command you typed running in the background. You can switch back to that command anytime by typing fg at the command prompt to bring the command back to the foreground.
Imagine you have a very important disk you want to mirror, or have a copy of. This could be a remote disk, like on a website, or even a directory.
You could copy and paste the files, at least if they are local disks, or use FTP. Problem solved, right?
Now, imagine you have two disks, both of which have data, but you need them to match. This was the problem I ran into with over 100 GB of data. I solved this in ubuntu using rsync.
To install, use apt-get:
$ sudo apt-get install rsync
As always, help is available by entering rsync –help. I wanted to sync two drives I use for music, both containing files the other didn’t and I wanted to keep the newest files. Here are the commands I used:
$ rsync -avuz /media/MUSIC2/ /media/MUSIC/
$ rsync -avuz /media/MUSIC/ /media/MUSIC2/
And that’s it! The data was synced on both drives using the command line!
I recently wanted to setup an automated backup system for a company I purchase hosting through (who promptly shut off full backups), and wanted to do so using FTP to my home computer.
The first thing I did was setup vsftpd. This is an FTP daemon that provides the ftp service. After installing the daemon and creating the system account with /bin/false as a shell, I edited /etc/vsftpd.conf. I didn’t run it on a non-standard port, although you can for greater security. I did NOT enable anonymous access and neither should anyone else not requiring them.
The next thing I did was set chown_uploads=YES and chown_username=<another user>. This will change the owner of the files uploaded to someone other than the ftp user. This way, if someone gets in, they can’t delete what’s in there.
Set the nopriv_user=<nonprivelaged user> and use that account and password to ftp. It’s sooo much safer. ftpd_banner will customize the banner displayed on successful login. I also chroot_local_user to keep the user in their own home directory. You can use the options below (chroot_list_enable=YES and chroot_list_file=<file>) to not jail a list of users. Jailing means they cannot traverse below their home directory.
All was well until someone started trying to hack my ftp service. They were trying lame user accounts like Administrator and Bill (no..it’s not Windoze). So I also installed fail2ban. This program monitors your access logs for several services and uses iptables to ban the offender from connecting to the server. It’s pretty sweet and very easy to install with Ubuntu!
Jeff pointed me to an article, supposedly breaking news, that described a “new” “web 2.0″ threat, trojans using XML feeds as command and control, and using social networking sites, or other “web 2.0″ sites like MySpace to store stolen information.
Hyped much? When you get down to it, RSS, xml-rpc…they’re merely calling a web page using a standard and getting an XML document. And MySpace is NOT a web 2.0 website, if there even is such a thing. I can’t stand the buzz word “web 2.0.” Just say ajax with standards compliance! =) Both of which, MySpace has not (last I checked anyway). “Web 2.0″ (yuck) is clean, bubbly, and fast (ajax)…again, definitely not MySpace.
But I digress. This security company is claiming these are new threats, when really, calling a web page has been around since the beginning of the internet. It is, after all, the nature of the internet. I assume the reason trojan makers have not used this method of receiving commands via the web (if they haven’t) is because it’s not encrypted.
So, for this to work, and be new, the trojan would have to be able to publish information using xml-rpc on an legitimate site. Is this possible?
Afterall, websites have been compromised for quite a while now. xml-rpc publishing requires authentication (or they’d better), so the trojan would have to hack the site at that point (not new), and the website would have to be widely published. The trojan would also have to either pass the commands/data in clear text, or encrypt it on their side.
If the commands were passed in clear text, the RSS feed being read by the trojan would show the command in clear text. If not, it would be gobbly gook. The only other option I see would be to translate legitimate looking headlines into commands…easily broken and noticeable on most sites.
So what exactly is “new” here? Well, the article claims trojans can bypass security metrics (i.e. port blocking and monitoring) by transmitting information to regularly used ports. So:
This method would have always worked for receiving commands. What about sending them? Compromised scripts (PHP, ASP) would do the trick..or compromised sites hosting those scripts.
But syndication (think “Web 2.0″) and xml-rpc is not open to my knowledge…you can’t just call an xml-rpc page and post content without authentication. And if you can…that needs to be fixed. So your trojan would have to have compromised the server…also not new.
If you stop the treat at the server (again, not new), this model doesn’t work. I see what they are saying, it is possible for 500 sites to be syndicating an RSS feed from a compromised website, possibly syndicating malicious commands, code etc…but it’s pretty far fetched and isn’t really an unstoppable, doom and gloom, can’t beat it model. If something like this ever took hold, an engine would have to simply look at where the xml feed was hosted, and block communication with that website, notifying the administrator of the website, right? But I don’t see a website administrator overlooking strange RSS feeds or weird syndicated content. Or am I missing something?
Here are some simple commands to use to get the disk usage in Linux.
$ df
will give you the disk usage for all mounted drives, including available and percentage of use.
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda2 17077684 7868596 8341580 49% /
varrun 1038144 272 1037872 1% /var/run
varlock 1038144 0 1038144 0% /var/lock
udev 1038144 84 1038060 1% /dev
devshm 1038144 0 1038144 0% /dev/shm
lrm 1038144 35324 1002820 4% /lib/modules/2.6.22-14-386/volatile
/dev/sdb2 64982632 23791484 37890196 39% /home/BigMama
/dev/sdb1 51199120 7619456 43579664 15% /home/windows
Use with the -h option for friendlier output.
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 17G 7.6G 8.0G 49% /
varrun 1014M 272K 1014M 1% /var/run
varlock 1014M 0 1014M 0% /var/lock
udev 1014M 84K 1014M 1% /dev
devshm 1014M 0 1014M 0% /dev/shm
lrm 1014M 35M 980M 4% /lib/modules/2.6.22-14-386/volatile
/dev/sdb2 62G 23G 37G 39% /home/BigMama
/dev/sdb1 49G 7.3G 42G 15% /home/windows
If you’re trying to track down a large file or folder, use the du command (again, you can use the -h option). This will list all files in the directory you’re in. If you just want the first level of folders (handy when you’re walking a file system), use –max-depth=1.
$du -h
616K ./cooper/orig
3.5M ./cooper/cropped
14M ./cooper
148K ./Photos/2006/11/24
3.3M ./Photos/2006/11/11
152K ./Photos/2006/11/26
448K ./Photos/2006/11/20
1.7M ./Photos/2006/11/12
812K ./Photos/2006/11/25
6.4M ./Photos/2006/11
3.5M ./Photos/2006/8/13
28K ./Photos/2006/8/6
6.5M ./Photos/2006/8/19
1.7M ./Photos/2006/8/14
964K ./Photos/2006/8/27
13M ./Photos/2006/8
6.0M ./Photos/2006/9/4
68M ./Photos/2006/9/23
2.7M ./Photos/2006/9/18
17M ./Photos/2006/9/8
30M ./Photos/2006/9/9
123M ./Photos/2006/9
784K ./Photos/2006/7/22
788K ./Photos/2006/7
156K ./Photos/2006/12/20
300K ./Photos/2006/12/12
460K ./Photos/2006/12
143M ./Photos/2006
143M ./Photos
166M .
$ du -h –max-depth=1
14M ./cooper
143M ./Photos
166M .
The . means your current folder, wherever you are in the directory structure. Remember to use sudo if you’re looking at system files.
The -h option can also be used with ls -l which will give you the directory listing, with other information including the size in human readable format. So:
$ ls -lh
total 9.4M
drwxr-xr-x 4 jsteel jsteel 4.0K 2007-12-06 22:22 cooper
-rw-r–r– 1 jsteel jsteel 1.6M 2007-11-01 20:13 cooper-png-jpg.zip
-rw-r–r– 1 jsteel jsteel 7.8M 2007-11-01 20:09 cooper.tar.gz
-rw-r–r– 1 jsteel jsteel 48K 2007-01-15 07:09 digikam3.db
drwxr-xr-x 3 jsteel jsteel 4.0K 2007-01-12 19:55 Photos
I switched my desktop back to gnome (think Ubuntu not Kubutnu), mostly because of AWN which is a cool menu with nice applets that extend the functionality (I have a system monitor, a weather applet, desktop switcher, file stacker, etc) (many examples can be seen on YouTube. AWN is still in heavy development so a few of the applets don’t work and there are some minor bugs, but very cool.
The gnome desktop manager seems to work better with the ubuntu base than the KDE desktop, although I miss a couple of features in KDE (like the popup with the file preview…although that may be a gnome setting).
This is one area where Linux shines..it gives you freedom. This is something few other usable operating systems (perhaps no other operating systems) can do. If you don’t like the desktop, change it or remove it, and do so with two commands at the (I know, scary..gulp) command line. Heck, if you really don’t like it and you know what you’re doing, you can edit and compile the source, creating something completely unique. If you’re a g33k and not using Linux, you’re…well…not a g33k.
Switching from kde to gnome entailed:
install gnome
$ sudo apt-get install ubuntu-desktop
During the setup, it may ask if you want this as the default. You can say yes, or no. You will be able to choose your session when you reboot, so you can actually have, and use both, but some things may be weird.
Now, either reboot, or stop, then start x
$ sudo /etc/init.d/x11-common restart
For the sake of completeness, I removed KDE
$ sudo apt-get remove kubuntu-desktop
And I’m done. New look, new tools. I can still use the tools found in KDE (ktorrent, k3b), even when I’m running gnome.
I have been struggling with copying a DVD in Linux. I like to copy the ones I purchase in case they get damaged.
In any case, copying a dvd in kubuntu (or ubuntu) requires the installation of libdvdcss2, which isn’t easily found.
I finally found the answer here:
https://help.ubuntu.com/community/DVD::Rip
Using the following lines, the library is installed and K3b (and other applications) can read an encrypted DVD.
sudo apt-get install libdvdread3 debhelper fakeroot
sudo /usr/share/doc/libdvdread3/examples/install-css.sh
# or if that last line didn’t work
sudo /usr/share/doc/libdvdread3/install-css.sh
As you can see, libdvdcss2 is downloaded and compiled using a script. It’s not available in the repositories due to legal issues. It’s illegal to copy an encrypted DVD because of widespread piracy. I believe I’ve read that it’s ok to make a backup copy of DVDs you’ve purchased.
**update**
I still cannot copy some DVDs with k9copy as it seems to fail in constructing the menus for the DVD.