X-Git-Url: https://git.mdrn.pl/pylucene.git/blobdiff_plain/a2e61f0c04805cfcb8706176758d1283c7e3a55c..aaeed5504b982cf3545252ab528713250aa33eed:/lucene-java-3.5.0/lucene/contrib/misc/src/java/overview.html?ds=sidebyside
diff --git a/lucene-java-3.5.0/lucene/contrib/misc/src/java/overview.html b/lucene-java-3.5.0/lucene/contrib/misc/src/java/overview.html
new file mode 100644
index 0000000..bd87055
--- /dev/null
+++ b/lucene-java-3.5.0/lucene/contrib/misc/src/java/overview.html
@@ -0,0 +1,77 @@
+
+
+
+
+ miscellaneous
+
+
+
+
+Misc Tools
+
+The misc package has various tools for splitting/merging indices,
+changing norms, finding high freq terms, and others.
+
+DirectIOLinuxDirectory
+
+
+NOTE: This uses C++ sources (accessible via JNI), which you'll
+have to compile on your platform. Further, this is a very
+platform-specific extensions (runs only on Linux, and likely only on
+2.6.x kernels).
+
+
+DirectIOLinuxDirectory is a Directory implementation that bypasses the
+OS's buffer cache for any IndexInput and IndexOutput opened through it
+(using the linux-specific O_DIRECT flag).
+
+
+Note that doing so typically results in bad performance loss! You
+should not use this for searching, but rather for indexing (or maybe
+just merging during indexing), to avoid evicting useful pages from the
+buffer cache.
+
+See here
+for details.
+
+Steps to build:
+
+ - cd lucene/contrib/misc/src/java/org/apache/lucene/store
+
+
- Compile NativePosixUtil.cpp -> libNativePosixUtil.so. On linux, something like gcc -fPIC -o libNativePosixUtil.so -shared -Wl,-soname,libNativePosixUtil.so -I$JAVA_HOME/include -I$JAVA_HOME/include/linux NativePosixUtil.cpp -lc -lstdc++. Add -m64 if you want to compile 64bit (and java must be run with -d64 so it knows to load a 64bit dynamic lib).
+
+
- Make sure libNativePosixUtil.so is on your LD_LIBRARY_PATH so java can find it (something like export LD_LIBRARY_PATH=/path/to/dir:$LD_LIBRARY_PATH, where /path/to/dir contains libNativePosixUtil.so)
+
+
- ant jar to compile the java source and put that JAR on your CLASSPATH
+
+
+
+To use this, you'll likely want to make a custom subclass of
+FSDirectory that only opens direct IndexInput/Output for merging. One
+hackish way to do this is to check if the current thread's name starts
+with "Lucene Merge Thread". Alternatively, you could use this Dir as
+is for all indexing ops, but not for searching.
+
+
+NativePosixUtil.cpp/java also expose access to the posix_madvise,
+madvise, posix_fadvise functions, which are somewhat more cross
+platform than O_DIRECT, however, in testing (see above link), these
+APIs did not seem to help prevent buffer cache eviction.
+
+
+