Annually Release Procedures

This page describes an annually release procedures for Lexical tools package with new set of lexicon data.

  1. Prepare lvg${YEAR} baseline
    • Copy lvg${PREV_YEAR} to lvg${YEAR}
      > cp -rp $LVG/lvg${PREV_YEAR} $LVG/lvg${YEAR}
    • Change ${PREV_YEAR} to ${YEAR} in build.xml files under $LVG, $LVG/examples, and $LVG/install
    • Change ${PREV_YEAR} to ${YEAR} in $LVG/overview.html
    • Update $LVG/data/config/lvg.properties, lvg.properties.TEMPLATE

      => Try to build with

      > ant release
      (should be OK to build)

  2. Update Java source code
    • Modify prolog of java files to
      • Remove all SCRs-XX from history tag
      • Modify V-${PREV_YEAR} from version tag
      		> cd ${LVG}/Components/BaselineCode/bin
      		> ModifyLvgJavaCode
      		> YYYY (${YEAR})
      		> 1
      		> y
      		

      => go to ${LVG}/${SOURCES}

      		> fgrep SCR */*.java
      		=> no SCR should exist in the source code
      		

      => build and test to make sure the result is same as last release

      		> cd ${LVG}
      		> ant clean
      		> ant release
      		> cd ..
      		> gtar -czvf lvg${YEAR}.tgz lvg${YEAR}
      		> install to ${PROJECT}
      		> The installation will not be successful because of the name of jar file.
      		> Manually change script in ${VLG}.${YEAR}/bin/
      		

  3. Build the unit test suite

    Please see Regression test on Flows Unit pages

  4. Update DB
    • Update HyperSonic JDBC Driver:
      • Download latest version of:
        • HyperSql Db

          copy 3 files to ${LVG}/lib/jdbcDrivers/HSqlDb/

          • hsqldb.jar
          • hsqldb_lic.txt
          • hypersonic_lic.txt
    • Compile:
      • > cd $LVG_DIR/loadDb
      • > ant clean
      • > ant cleanJar
      • > ant
    • Load previous data into latest HyperSonic Sql Db:
      • > vi $LVG_DIR/loadDb/sources/gov/nih/nlm/nls/lvg/loadDb/HSqlDb/db.cfg
        • DB_NAME = lvgXXXX
        • LVG_DIR = /export/home/lu/Development/LVG/lvgXXXX
      • > cd $LVG_DIR/loadDb/bin
      • > ./2.LoadDb ${YEAR}
        => Source codes might need to be modified (on SQL State and Vendor Error code) in case of dropping non-exist table if HsqlDb is upgraded with new SQL/JDBC standard
        		DbBase.ExecuteDdl(String query)
        		...
        		if((e.getSQLState().equals("42501"))
        		&& (e.getErrorCode() == -5501))
        		...
        		
      • add readonly=true in $LVG_DIR/data/HSqlDb/lvg${YEAR}.properties
        => This is a must to make HSqlDb multi-threads
    • Test:
      • > cd $LVG_DIR/loadDb/bin
      • > ./3.TestDb ${YEAR}

  5. Update Installation Program
    • Change program parameters:
      • Download latest version of:
        • Java: Windows (offline installation), Solaris (32-bit), Solaris x 86, Linux (self-extracting file)
      • > cd ${LVG_DIR}/install/sources/gov/nih/nlm/nls/lvg/install/Setup
      • Update all data members in Param.java
      • Update all scripts in install/bin
        • Update lvg${YEAR}
        • Update Java version
    • Compile:
      • > cd ${LVG_DIR}/install
      • > ant clean
      • > ant cleanJar
      • > ant

      • > cd ${LVG_DIR}
      • > ant dist
    • Test:
      • > cd $LVG_DIR
      • > ./install/bin/install_test.sh

      • Change JAVA=java for all *_test
      • Change name for all *_test to *
        • > mv lgt_test lgt
        • > mv luiNorm_test luiNorm
        • > mv lvg_test lvg
        • > mv norm_test norm
        • > mv toAscii_test toAscii
        • > mv wordInd_test wordInd
        • > mv fields_test fields

      Pack and perform unit test again to complete the baseline

      	> cd ${LVG}
      	> ant clean
      	> ant release
      	> pack lvg${YEAR}.tgz
      	> install to ${PROJECT}
      	> perform Unit test
      	=> Update config file: lvg.properties.hsql
      	> Make a copy of this baseline
      	

  6. Complete SCRs for ${YEAR} release

  7. Integrate with New Lexicon Data

  8. Update other software components

  9. Test Procedures
    • ASCII test: make sure no non-ASCII characters in the output of norm and luiNorm:
      shell> ${TEST}/LVG/ComponentTest/Ascii/bin
      shell> TestAsciiOnNorm ${YEAR}
      6
      
      => The word count of all four *.2.out fiels should be 0
      

    • Unit test:
      => ${TEST}/LVG/UnitTest/bin/

    • Deploy test result to web site document
      shell> cd /export/home/lu/Development/LVG/PostProc/TestExamples/bin
      shell> cd /export/home/lu/Development/LVG/PostProc/UnicodeTables/bin
      
    • Release test on different platforms
      • Linux
      • PC

    • This is the Internal release for OCSS.

    • Change "\t" to "    " for all Java code under $LVG_DIR/sources/gov/nih/nlm/nls/lvg/
      shell> cd ${LVG}/Components/BaselineCode/ModifyLvgJavaCode
      
    • Performance test on norm and luiNorm:
      • Preparation:
        shell> cd ${TEST}/LVG/PerformanceTest/data/
        shell> mkdir ${YEAR}
        shell> cd ${YEAR}
        shell> mkdir Input
        shell> ln -sf ../../Input/test.in test.in
        shell> ln -sf ../../Input/test.in.100 test.in.100
        shell> ln -sf ../../Input/test.in.10000 test.in.10000
        shell> ln -sf /export/home/lu/Development/Lexicon/data/2007/outputs/lexiconTerms.data lexiconTerms.data
        shell> ln -sf /export/home/lu/Development/Meta/data/2006/meta.data meta.data
        shell> cp -rp ../../${YEAR}/Input/script .
        shell> edit * (change ${YEAR-1} to ${YEAR})
        
      • Run Test:
        shell> cd ${TEST}/LVG/PerformanceTest/bin/
        shell> TestAll
        	${YEAR}
        

  10. Lvg Compile & Pack
    • > cd $LVG_DIR
    • > ant clean
    • > ant
    • > ant release

    • pack: gtar -czvf lvg${YEAR}.tgz lvg${YEAR}
    • unpack: gtar -xzvf lvg${YEAR}.tgz

  11. Update Web Tools
    • Copy $WEB_LVG/${PREV_YEAR} to $WEB_LVG/${YEAR} ...