How to design an external prefilter library

The prefilter library must be implemented in C/C++, and must:

include the prefilter interface definition header file, extpfapiv1.h.
implement the a_text_source interface.
provide an entry point function that initializes and returns an instance of a_text_source (prefilter) and the label of the character set supported by the prefilter.

Calling sequence for the prefilter

The following calling sequence is executed by the consumer of the prefilter for each document being processed:

begin_document(a_text_source*)
get_next_piece(a_text_source*, buffer**, len*)
get_next_piece(a_text_source*, buffer**, len*)
...
end_document(a_text_source*)

Note

end_document can be called multiple times without an intervening begin_document call. For example, if one of the documents to be indexed is empty, the database server may call end_document for that document without calling begin_document.

The get_next_piece function should filter out the unnecessary data such as formatting information and images from the incoming byte stream and return the next chunk of filtered data in a self-allocated buffer.

How to design an external prefilter library

Calling sequence for the prefilter

Note

External prefilter library workflow

See also