Friday, May 20, 2011

Trick: gather outbound GETs made by Nexus for a $reason

The $reason might differ a lot, I was just curious how to do this in a "lab" environment, to have a list of URLs fetched by Nexus (that were made actually to fulfill client requests). Again, this is a test, not quite usable in production environments -- unless you spice it up maybe.

All I wanted to have a list of URLs (artifacts) that my Nexus fetched during a test. I wanted to check that list, sort it, count the distinct URLs, check for dupes -- if any, etc. This is here just as reference to me in future, or maybe may help somebody else too.

How to do it:

  1. Set up a "clean" Nexus installation, by let's say unzipping the bundle somewhere.

  2. Fire it up, login as "admin" user and set logging to DEBUG level over UI -- Nexus will spit out outgoing HTTP GETs in DEBUG log level like these:
    jvm 1    | 2011-05-20 14:44:58 ... - Invoking HTTP GET method against remote location
  3. Start some client to fetch against nexus, I did this:
    cstamas@marvin test]$ mvn -s settings-1.xml clean install > b1.txt & mvn -s settings-2.xml clean install > b2.txt & mvn -s settings-3.xml clean install > b3.txt &
    ... and went for a coffee.
  4. process the logs.

Processing the logs

  1. Concat the logs into single file -- if needed. I had to, I ended up with two log files, since DEBUG made wrapper to roll the file based on size I guess.
  2. Filter the logs appropriately, I used combination of tools like grep and awk, to produce my list of URLs

Example session:

$ cp ~/worx/sonatype/nexus/nexus/nexus-distributions/nexus-oss-webapp/target/ .

$ unzip

$ cd nexus-oss-webapp-1.9.2-SNAPSHOT/bin/jsw/macosx-universal-32/

$ ./nexus console

$ cd ../../../logs

$ less wrapper.log

$ less wrapper.log.1

$ cat wrapper.log.1 wrapper.log > remoteFetches.txt

$ less remoteFetches.txt

$ cat remoteFetches.txt | grep "Invoking HTTP GET method against remote location" > remoteFetches-filtered.txt

$ less remoteFetches-filtered.txt

$ awk 'BEGIN{FS=" "}{ printf "%s\n", $18}' remoteFetches-filtered.txt > remoteFetches-urls.txt

$ less remoteFetches-urls.txt


It gave me list like this one (unsorted, URLs are ordered as Nexus made them):


Have fun!

No comments: