Using the delta index speeds up the update of the index, as long as the delta index is small. If it becomes too large, you have to integrate it into the main index. TREX then creates a new delta index after this integration.
The integration process involves TREX rewriting all main index files. The duration of the integration process depends on the size of the main index. It can last a few minutes or several hours.
In a distributed system the entire main index has to be replicated after the integration has taken place. This replication takes about the same amount of time as the initial replication.
The index server cannot index new documents during the integration of the delta index. This has the following effects:
If indexing takes place with a queue server, the queue server retains the documents until the integration process has been completed. Then the queue server transmits the documents to the index server.
If indexing takes place without a queue server, the application can continue to send indexing requests to the index server. However, the index server only processes them after the completion of the integration process. This means that it takes longer for indexing requests to be processed and for the application to receive the relevant response.
You can trigger the integration process manually or carry it out at defined time intervals. There are two difference procedures for time-dependent integration. The procedure that you use depends on whether indexing takes place with or without a queue server (QS). The table below gives an overview of the procedures.
Use with |
||
Procedure |
Indexing with QS |
Indexing without QS |
Manual |
|
|
Time-dependent using the queue server |
|
|
Time-dependent using the Python scheduler |
|
We recommend the following for the time of the integration:
· Trigger the first integration process if the delta index is bigger than 500 MB. You can find out the size of the delta index in the window
in the TREX admin tool.· The integration process should take place at times when the system is not too busy.
· Do not carry out the integration process too often. With large indexes, the integration and subsequent replication of the main index takes a corresponding amount of time.
Integrating the Delta Index Manually
Go to the window
in the TREX admin tool.Select the index in question and choose Merge Delta Index.
Integrating the Delta Index Time-Dependently Using the Queue Server
In the queue parameters, enter the time for the integration in Merge Time for Delta Index.
Use All (4:00) to trigger integration every morning at 4am.
You do not need to coordinate the integration time with other activities carried out by the queue server and index server. If the activities collide, the index server coordinates when it carries out which action.
For more information on changing queue parameters, see Configuring Queue Parameters.
Integrating the Delta Index Time-Dependently Using the Python Scheduler
Change the following configuration files on all master name servers:
Configuration File |
Change |
TREXDaemon.ini |
|
crontab.ini |
Remove the comment sign from the following line: <schedule> python mergeDeltaIndex.py silent allIndexes=1 '' Modify the schedule if necessary. For information on syntax and for examples, see the configuration file. |