Skip to main content

Nessus XML Validation

Tenable Network Security no longer provides a DTD for Nessus 4.2 reports. I'm working on adding Nessus support to OpenFISMA, and wanted to be able to validate that the XML that I was processing from a user was in fact the correct format. I ended up using Instance2Scehma to generate the RNG schema, and then used Trang to generate other schemas that might be useful to other people.

So, XMLReader in PHP is a little bit weird, and you can't validate the XML until you start reading it. And XMLReader won't throw validation exceptions, so you can't put your reading into a try/catch and stop processing once you find out that there's an error. Bummer. What you can do though, is do your processing, check for errors, and then persist the data after checking to make sure that the XML is valid.

<?php
$xmlReader = new XMLReader();
$xmlReader->open('report.nessus');
$xmlReader->setRelaxNGSchema('schema.rng');

while($xmlReader->read()) {
  // process XML ...
}

if($xmlReader->isValid()) {
  // persist data
} else {
  // handle invalid XML
  throw new Exception('Bad XML.');
}
?>

Schema files are attached in a variety of formats at the bottom of this post.

How-to: Upgrade Debian 4 to Debian 5

Posted in

One of the server providers that I use for some reason only likes to deploy Debian 4, even though 5 is out and stable. Here's how to easily upgrade from Etch to Lenny via SSH/command line.

Edit /etc/apt/sources.list:

deb <a href="http://ftp.us.debian.org/debian/" title="http://ftp.us.debian.org/debian/">http://ftp.us.debian.org/debian/</a> etch main contrib non-free
deb-src <a href="http://ftp.us.debian.org/debian/" title="http://ftp.us.debian.org/debian/">http://ftp.us.debian.org/debian/</a> etch main contrib non-free

deb <a href="http://security.debian.org/" title="http://security.debian.org/">http://security.debian.org/</a> etch/updates main contrib non-free
deb-src <a href="http://security.debian.org/" title="http://security.debian.org/">http://security.debian.org/</a> etch/updates main contrib non-free

deb <a href="http://volatile.debian.org/debian-volatile" title="http://volatile.debian.org/debian-volatile">http://volatile.debian.org/debian-volatile</a> etch/volatile main contrib non-free
deb-src <a href="http://volatile.debian.org/debian-volatile" title="http://volatile.debian.org/debian-volatile">http://volatile.debian.org/debian-volatile</a> etch/volatile main contrib non-free

Then do the following:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade

After this completes, edit /etc/apt/sources.list and replace all instances of "etch" with "lenny":

deb <a href="http://ftp.us.debian.org/debian/" title="http://ftp.us.debian.org/debian/">http://ftp.us.debian.org/debian/</a> lenny main contrib non-free
deb-src <a href="http://ftp.us.debian.org/debian/" title="http://ftp.us.debian.org/debian/">http://ftp.us.debian.org/debian/</a> lenny main contrib non-free

deb <a href="http://security.debian.org/" title="http://security.debian.org/">http://security.debian.org/</a> lenny/updates main contrib non-free
deb-src <a href="http://security.debian.org/" title="http://security.debian.org/">http://security.debian.org/</a> lenny/updates main contrib non-free

deb <a href="http://volatile.debian.org/debian-volatile" title="http://volatile.debian.org/debian-volatile">http://volatile.debian.org/debian-volatile</a> lenny/volatile main contrib non-free
deb-src <a href="http://volatile.debian.org/debian-volatile" title="http://volatile.debian.org/debian-volatile">http://volatile.debian.org/debian-volatile</a> lenny/volatile main contrib non-free

Now we go through the update/upgrade/dist-upgrade process again, and then reboot:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade
sudo shutdown -r now

How-to: Convert mysql to sqlite

Posted in

I had a need today to convert a mysql database into a sqlite3 database.

First, you need this script:

#!/bin/bash
if [ "x$1" == "x" ]; then
  echo "Usage: $0 <dumpname>"
  exit
fi

cat $1 |
grep -v ' KEY "' |
grep -v ' UNIQUE KEY "' |
grep -v ' PRIMARY KEY ' |
sed '/^SET/d' |
sed 's/ unsigned / /g' |
sed 's/ auto_increment/ primary key autoincrement/g' |
sed 's/ smallint([0-9]*) / integer /g' |
sed 's/ tinyint([0-9]*) / integer /g' |
sed 's/ int([0-9]*) / integer /g' |
sed 's/ character set [^ ]* / /g' |
sed 's/ enum([^)]*) / varchar(255) /g' |
sed 's/ on update [^,]*//g' |
sed 's/\\r\\n/\\n/g' |
sed 's/\\"/"/g' |
perl -e 'local $/;$_=<>;s/,\n\)/\n\)/gs;print "begin;\n";print;print "commit;\n"' |
perl -pe '
if (/^(INSERT.+?)\(/) {
  $a=$1;
  s/\\'
\''/'\'\''/g;
  s/\\n/\n/g;
  s/\),\(/\);\n$a\(/g;
}
' > $1.sql
cat $1.sql | sqlite3 $1.db > $1.err
ERRORS=`cat $1.err | wc -l`
if [ $ERRORS == 0 ]; then
  echo "Conversion completed without error. Output file: $1.db"
  rm $1.sql
  rm $1.err
else
  echo "There were errors during conversion.  Please review $1.err and $1.sql for details."
fi

Then, dump a copy of your database:

josh@josh-laptop-work:~/tmp$ mysqldump -u root -p --compatible=ansi --skip-opt generator > dumpfile

And now, run the conversion:

josh@josh-laptop-work:~/tmp$ mysql-to-sqlite.sh dumpfile

And if all goes well, you should now have a dumpfile.db which can be used via sqlite3.

josh@josh-laptop-work:~/tmp$ sqlite3 dumpfile.db
SQLite version 3.6.10
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> .tables
dg_cities                 dg_forms                  dg_surnames            
dg_counties               dg_provinces              dg_user_accounts        
dg_countries              dg_provinces_netherlands
dg_first_names            dg_states

So, MyPhotoAlbum.com acquired some of the Photrade.com assets ...

So, as most people know, I was one of the developers who worked on Photrade up until we ran out of money, and couldn't find anymore funding. One of the biggest things I did there was build out the YUI based photo management interfaces, and I built out the entire server infrastructure. The site has apparently been limping around with no active development since October.

To my surprise, I received this in my spam folder today:

Dear Photrade Users,

At the end of 2008 you were informed by the owners of Photrade.com that they had lost their funding and would not continue to develop the website. And at some point earlier this year, all of the full sized images that had been uploaded to Photrade were deleted. I hope you kept a backup copy of your images.

Last month, MyPhotoAlbum Inc. acquired some of the Photrade.com assets. We will be adapting some of the Photrade.com technology to enhance the MyPhotoAlbum.com service. Please note that we also own and operate dotPhoto.com.

We invite you to try the MyPhotoAlbum photo service and are pleased to offer you complimentary use of our Club Membership subscription service for one year - FREE. Simply sign up for a FREE MyPhotoAlbum account and then drop a line to our Support Team at <a href="http://support.myphotoalbum.com" title="http://support.myphotoalbum.com">http://support.myphotoalbum.com</a>, quoting reference PHOTRADE to receive your free upgrade.

Please note that we will be announcing the MyPhotoAlbum Pro Service within a month that will include all of the Photrade services and more!

If you have any questions please do not hesitate to contact me at [email protected]

Thanks,
Peter Macnee
CEO, MyPhotoAlbum Inc.

Well, isn't that interesting. I've never even heard of MyPhotoAlbum. Oh, and I hold equity in Photrade. But that doesn't really matter, because I'm sure any money that was made from selling off these assets went to the original investors and/or to payoff the enormous amount of debt that Photrade has racked up keeping the site running since October. Those servers at EC2 are expensive, and I was laid off before given a chance to scale back the server infrastructure to keep things afloat longer. Whoops. Brilliant move by the original investors/board of advisors/CEO. Let's lay everybody off to scale back our benefit and salary costs, but not try to scale back any other costs.

Regardless, I'm guessing the reason that MyPhotoAlbum bought "some assets" instead of the whole company, is because Photrade is probably going to file for bankruptcy, to get out of the debt that's been racked up. And nobody wants to buy debt, obviously.

Anyways, there's a particular part of the email that is interesting to me.

And at some point earlier this year, all of the full sized images that had been uploaded to Photrade were deleted. I hope you kept a backup copy of your images.

This, my friends, is completely false. All of those images are STILL there. Half a terrabyte plus of images are still laying around on EC2. They're not lost, the database isn't corrupt, the images aren't corrupt. Either somebody was lied to in this deal, somebody is dumb and doesn't know what they're talking about, or Photrade/MyPhotoAlbum doesn't feel like spending the half hour to build a script to allow Photrade users to download their original photos. Which, by the way, if Peter is reading this, I'd be more than happy to build such a script/page for free. Our users were awesome, it just sucks that the company was so horribly managed, for all parties involved.

Anyways, I sent an e-mail to Peter, no idea if he'll respond or not, letting him know that the images actually still exist at this moment. I'll probably get sued by somebody over this post, I think my NDA still applies for a few more months. But hey, I'm a poor man right now, and I think that Photrade is probably just as poor. If the company wants to sue me for some debt, awesome, I'll give it up, no problem.

HAPPY CAPSLOCK DAY

CLASS DUMBCLASS
{
  PUBLIC STATIC FUNCTION CAPS(&$S)
  {
    $S = STRTOUPPER($S);
  }
}

$CRAP = "hello";
DUMBCLASS::CAPS($CRAP);
ECHO $CRAP;

How-to: Replace Keys of an Array in PHP

Posted in

I had a need tonight to replace the keys in an array in PHP. I couldn't find a good solution on any mailing lists or other sites, so I thought I'd share the class that I came up with, and it's test.

First up, the test:

<?php
require_once 'PHPUnit/Framework.php';
require_once 'ArrayHelper.class.php';

class ArrayHelperTest extends PHPUnit_Framework_TestCase
{
  public function testRenameKeys()
  {
    $keys   = array('newkey1',
                    'newkey2',
                    'newkey3');
                   
    $array  = array('oldkey1' => 'value1',
                    'oldkey2' => 'value2',
                    'oldkey3' => 'value3');
                   
    ArrayHelper::renameKeys($array,$keys);
   
    $this->assertArrayHasKey($keys[0],$array);
    $this->assertArrayHasKey($keys[1],$array);
    $this->assertArrayHasKey($keys[2],$array);
    $this->assertEquals(3,count($array));
  }
}

Next, the class itself, after the jump.

Frozen Food Review: Michelina's Budget Gourmet Classics Pasta & Chicken

Today I made the decision to stop ordering take-out, and stock up on frozen/instant food to save some money instead of dropping it all on take-out food that ends up all tasting the same anyways. In this light, I've decided that I'll also review the frozen foods that I choose to consume so that other programmer types who have no time to deal with the hassle of cooking (yes, I know how to cook) can make educated decisions on what frozen foods to eat/avoid.

The first meal in this review is a Pasta and Chicken dish from Michelina's, from the "Budget Gourmet Classics" series.

Text of Mantasano's Article on Detais of Kaminsky's DNS Attack

Unless you've been living under a rock today, I'm sure you've been hearing about the Matasano blog leak of the details of Kaminsky's DNS attack. It was ripped off the site, and most other sites who've had it posted up have had it ripped down as well. I'm posting the story here in it's entirety, simply because it's interesting, and I think everyone has a right to know. Regardless, all of the vendors have released patches for this vuln already. If the ISPs haven't patched yet, it's on them, not Kaminsky, Matasano, ecopeland, or me for reposting it.

Enjoy.

How Software Companies Die by Orson Scott Card

The environment that nutures creative programmers kills management
and marketing types - and vice versa. Programming is the Great Game.
It consumes you, body and soul. When you're caught up in it, nothing
else matters. When you emerge into daylight, you might well discover
that you're a hundred pounds overweight, your underwear is older than
the average first grader, and judging from the number of pizza boxes
lying around, it must be spring already. But you don't care, because
your program runs, and the code is fast and clever and tight. You won.
You're aware that some people think you're a nerd. So what? They're
not players. They've never jousted with Windows or gone hand to hand
with DOS. To them C++ is a decent grade, almost a B - not a language.
They barely exist. Like soldiers or artists, you don't care about the
opinions of civilians. You're building something intricate and fine.
They'll never understand it.

New Toy --- ASUS eeePC 4G Galaxy Black

Posted in

It's been awhile since my last post, been busy working 100+ hour weeks at a startup.

Anyways, I picked up an ASUS eeePC 4G in Galaxy Black last week, along with 2GB of ram to juice it up. The default linux install didn't really impress me, so I've wiped it out and install eeexubuntu on it, which runs pretty well. So far I'm impressed by it, I did hack the keymaps so that shift was in the proper place on the keyboard. Next things I'm looking at doing are hardwiring two 32GB USB flash drives inside of it, and setting them up in software RAID 0, to expand storage from 4GB to 68GB or so. Might go completely crazy and drop 3 of them in and do RAID 5 ;) I'm also considering wiring up my ATT USBConnect card, however, it appears linux support is lacking, so if I go that route I'll have to see if I can load OSX onto it first. I'm not really very excited about the prospect of loading Windows onto the eee, but we'll see what happens.

I've also "overclocked" the processor from 633MHz to 900MHz, which is it's native speed. Apparently ASUS felt it was a good idea to downclock, I felt otherwise obviously. This can be done with the eee.ko kernel module via software, which also allows temperature monitoring and manual fan control.

Syndicate content