Frequently Asked Questions

(Please read before asking a question)

  • How can I ask a question?
    See Contact Us

  • I can't install lexical tools successfully?
    One of the most common mistakes is that users install lexical tools from the wrong directory. Make sure to run the lexical tools installation script from the top directory of $LVG_DIR. Please refer to installation instruction for details.

  • Problems of using my old Java code with lvg.2007 (and later) Java APIs?
    Lvg was developed and compiled in Java 1.5 in 2006, 2007, and 2008; in Java 1.6 in 2009 and 2010. You will need to use JDK 1.5/1.6 to compile your java codes and run your applications on JRE 1.5 and JRE 1.6. Please refer to Java 1.5 upgrade notes for details.

  • What are the JRE version vs. Lexical Tools version and embedded database?
    Lexical Tools VersionJRE VersionEmbedded Database
    20021.2IDB V3.26
    20031.4IDB V3.26
    20041.4IDB V3.26
    20051.4.2.05HSqlDb 1.7.2
    20061.5.0.02HSqlDb 1.8.0.01
    20071.5.0.07HSqlDb 1.8.0.04
    20081.5.0.11HSqlDb 1.8.0.07
    20091.6.0.06HSqlDb 1.8.0.10
    20101.6.0.14HSqlDb 1.8.0.10
    20111.6.0.21HSqlDb 2.0.0

    Please refer to Java 1.5 upgrade notes for details.

  • What is the difference between lvg2010 and lvg2010lite?
    See lvgLite documents

  • Can we use other database instead of the default database (HSqlDb) for Lexical tools?
    Yes, all types of database can be used with Lexical Tools as long as there is a JDBC connector for it. MySql database is used as an example for illustration. Please refer to install MySql database option for details.

  • How do we use Lexical tools APIs?
    Please refer to Lexical Tools APIS for details.

  • How to make sure it is thread-safe when using Lexical in the application?
    Have separate thread to create separate Lexical Tools APIs in the applications.

  • Is there any tool in Lexical Tools for converting Unicode to ASCII?
    Yes, a new tool, toAscii, is provided since lvg.2009 release.

  • Is there any tool in Lexical Tools for cutting out and rearranging fields?
    Yes, a new tool, fields, is provided since lvg.2011 release.

  • Which lvg flow components can be used for converting Unicode to ASCII?
    Lexical tools provide several options for Unicode to ASCII operations. Flows of -f:q5 and norm (-f:N) normalize Unicode to pure ASCII. Flows of -f:q, -f:q0, -f:q1, -f:q2, -f:q3, -f:q4, -f:q7, -f:q8 provide other useful Unicode normalizations.

  • Are there any scripts that we can use to run lvg or Norm?
    Yes, scripts of norm, luiNorm, wordInd, lvg, toAscii, fields, and lgt are generated under "${LVG_DIR}/bin/" after normal installation. This directory includes scripts for Unix and batch files for Windows, respectively.

  • Is the latest Java version faster than old C version?
    The answer is "Yes". The latest java version is as fast as old C version (or even faster). It's because we resolved the performance bottle neck by using HSqlDb and using much faster machine. For the first Java version of lvg.2002, it is slower than C version. The major bottle neck was the database (IDB) and persistent trie. We tried to run lvg on a faster database (MySql), the performance is improved dramatically. After lvg2003, we had improved the performance mainly on trie and some other code optimization. The performance of norm after version 2003 is at the same order of magnitude as C version on Solaris, Sparc platform.

    In lvg.2004, we improved performance (about 50%) by taking advantage of MySql V4.0, new JDBC driver, and other code optimization. After lvg.2005, Lvg uses HSqlDb as default DB to improve performance. As a matter of fact, lvg.2005 or later versions are faster than old C version.