Overview | Package | Class | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## fr.gouv.culture.sdx.oai
Class AbstractDocumentBaseOAIHarvester

    java.lang.Object
      fr.gouv.culture.util.apache.avalon.excalibur.concurrent.Semaphore
          fr.gouv.culture.util.apache.avalon.excalibur.concurrent.Mutex
              fr.gouv.culture.util.apache.cocoon.xml.AbstractSynchronizedXMLProducer
                  fr.gouv.culture.util.apache.cocoon.xml.AbstractSynchronizedXMLPipe
                      fr.gouv.culture.oai.SynchronizedOAIObjectImpl
                          fr.gouv.culture.oai.AbstractOAIHarvester
                              fr.gouv.culture.sdx.oai.AbstractDocumentBaseOAIHarvester

All Implemented Interfaces:
  EDU.oswego.cs.dl.util.concurrent.Sync, OAIHarvester, OAIObject, DocumentBaseOAIHarvester, Saveable, Target, SynchronizedXMLConsumer, SynchronizedXMLPipe, SynchronizedXMLProducer, org.apache.avalon.excalibur.pool.Poolable, org.apache.avalon.excalibur.pool.Recyclable, org.apache.avalon.framework.configuration.Configurable, org.apache.avalon.framework.context.Contextualizable, org.apache.avalon.framework.logger.LogEnabled, org.apache.avalon.framework.service.Serviceable, org.apache.cocoon.xml.XMLPipe, org.apache.cocoon.xml.XMLProducer, org.apache.excalibur.xml.sax.XMLConsumer, org.apache.excalibur.xml.sax.XMLizable, org.xml.sax.ContentHandler, org.xml.sax.ext.LexicalHandler

Direct Known Subclasses:
  LuceneDocumentBaseOAIHarvester

    public abstract class AbstractDocumentBaseOAIHarvester

extends AbstractOAIHarvester
implements DocumentBaseOAIHarvester

Nested Class Summary

Nested classes/interfaces inherited from interface fr.gouv.culture.sdx.oai.DocumentBaseOAIHarvester
`DocumentBaseOAIHarvester.ConfigurationNode`

Nested classes/interfaces inherited from interface fr.gouv.culture.oai.OAIObject
`OAIObject.Node`

Field Summary
`protected Database` | `_database` Underlying database to store any info
`protected org.apache.cocoon.serialization.XMLSerializer` | `cBytes`
`protected java.lang.String` | `defaultTransformerFactory`
`protected java.lang.String` | `defaultTransformerIndent`
`protected java.util.ArrayList` | `deletedDocs`
`protected DocumentBase` | `docbase` The underlying document base
`protected java.lang.String` | `docbaseId` Id of the underlying document base
`protected static java.lang.String` | `ERROR_CODE`
`protected java.io.FileOutputStream` | `fileOs`
`protected java.util.Hashtable` | `filesProperties` List OAI files with OAI properties
`protected boolean` | `forceIndexOnHarvestError` Force indexation on harvest error option Default: false.
`protected static java.lang.String` | `FORCEINDEXONHARVESTERROR`
`protected java.io.File` | `harvestDoc`
`protected IDGenerator` | `harvesterIdGen` IDGenerator for this object
`protected boolean` | `indexAtHarvestEnd` Indexation at the end of harvesting option Default: true.
`protected static java.lang.String` | `INDEXATHARVESTEND`
`protected boolean` | `keepDeletedRecords`
`protected boolean` | `keepHarvestedRecords` Force harvester to keep harvested records (default: false) Force harvester to keep harvested records (XML files) in file system server.
`protected java.util.Set` | `m_docsaddedIds`
`protected java.util.Set` | `m_docsdeletedids`
`protected java.util.Set` | `m_docsToDeleteIds`
`protected static java.lang.String` | `NO_DOCS_DELETED`
`protected static java.lang.String` | `NO_DOCS_HARVESTED`
`protected int` | `noDocsDeleted`
`protected int` | `noHarvestedDocs`
`protected int` | `noRecordsPerBatch`
`protected static java.lang.String` | `OAI_FAILED_HARVEST`
`protected static java.lang.String` | `OAI_FROM`
`protected static java.lang.String` | `OAI_HARVEST_ID`
`protected static java.lang.String` | `OAI_HARVESTER_LAST_UPDATED`
`protected static java.lang.String` | `OAI_HARVESTER_RESUMPTION_TOKEN`
`protected static java.lang.String` | `OAI_IDENTIFIER`
`protected static java.lang.String` | `OAI_METADATA_PREFIX`
`protected static java.lang.String` | `OAI_SET`
`protected static java.lang.String` | `OAI_UNTIL`
`protected static java.lang.String` | `OAI_VERB`
`protected org.apache.cocoon.xml.XMLPipe` | `oaiStripper`
`protected Pipeline` | `pipe` Pre-indexation pipeline
`protected TimeScheduler` | `scheduler` Time scheduler for stored requests
`protected java.util.Hashtable` | `storedRequests` Requests in application.xconf
`protected java.util.Hashtable` | `storeRepositoriesRefs` References to the underlying documentbase's/application's repositories
`protected java.io.File` | `tempDir`
`protected java.io.File` | `tempDirBatch`
`protected java.lang.String` | `tempDirPath` Directory to store harvested documents Temporary path of the directory where the harvested documents will be stored.
`protected java.lang.String` | `TEMPFILE_SUFFIX`
`protected static java.lang.String` | `TRANSFORMER_FACTORY`
`protected static java.lang.String` | `TRANSFORMER_INDENT`
`protected java.lang.String` | `transformerFactory` XML Transformer factory classe name.
`protected java.lang.String` | `transformerIndent` XML Transformer indent option.
`protected XMLDocument` | `urlResource`

Fields inherited from class fr.gouv.culture.oai.AbstractOAIHarvester
`adminEmails, captureElemContent, captureRecord, currentDatestamp, currentMetadtaUrlIdentifier, currentOaiIdentifier, currentOaiStatus, cursor, deleteRecord, errorCode, firstXmlConsumer, identifierName, manager, newRequestUrl, OAI_REPOSITORY_URL, OAI_REQUEST_URL, repoUrl, requestParams, requestUrl, responseDate, resumptionToken, sBuff, userAgent`

Fields inherited from class fr.gouv.culture.oai.SynchronizedOAIObjectImpl
`_context, logger`

Fields inherited from class fr.gouv.culture.util.apache.cocoon.xml.AbstractSynchronizedXMLProducer
`synchronizedXmlConsumer`

Fields inherited from interface fr.gouv.culture.oai.OAIObject
`HTTP_HEADER_NAME_FROM, HTTP_HEADER_NAME_USER_AGENT, NUMBER_RECORDS_PER_RESPONSE, STRING_DATEFORMAT_GRANULARITY_DAY, STRING_DATEFORMAT_GRANULARITY_SECOND`

Fields inherited from interface EDU.oswego.cs.dl.util.concurrent.Sync
`ONE_CENTURY, ONE_DAY, ONE_HOUR, ONE_MINUTE, ONE_SECOND, ONE_WEEK, ONE_YEAR`

Fields inherited from interface fr.gouv.culture.sdx.utils.save.Saveable
`ALL_SAVE_ATTRIB, PATH_ATTRIB, SAVE_DIRECTORY_PARAM`

Constructor Summary
`AbstractDocumentBaseOAIHarvester(DocumentBase base)` Basic constructor

Method Summary
`void` | `backup(SaveParameters save_config)` Save the timeStamp of the Harvester
`protected void` | `captureRecord()` Ends the capture of an oai record.
`protected void` | `captureResourceFromUrlIdentifier()` Captures the xml from a url taken from an oai record and adds it to the oai-record as a sibling of the element
`boolean` | `checkGranularity(java.lang.String granularity)` Check the granularity of an AOI provider : YYYY-MM-DDThh:mm:ssZ or YYYY-MM-DD
`void` | `close()` Close OAI harvester.
`void` | `configure(org.apache.avalon.framework.configuration.Configuration configuration)` OAI harvester configuration Configures the OAI harvester reading `application.xconf` file wich may contains a section such as: <sdx:documentBase [...]> <sdx:oai-harvester adminEmail="{some.body@some.where}" keepDeletedRecords="{true|false}" noRecordsPerBatch="{number}" transformer-factory="{Transformer factory classe name}" transformer-indent="{yes|no}" keepHarvestedRecords="{true|false}" tempDirPath="{directory path}"> <sdx:oai-data-providers> <sdx:oai-repository [...]>[...]
`protected void` | `configureAdminEmails(org.apache.avalon.framework.configuration.Configuration configuration)` Configures a list of admin emails can be sub-elements, a single attribute, or both
`protected void` | `configureDatabase(org.apache.avalon.framework.configuration.Configuration configuration)` Configures the internal database
`protected void` | `configureDataProviders(org.apache.avalon.framework.configuration.Configuration configuration)` Configures data providers info that can be reused and from which requests can be automatically executed
`protected void` | `configureHarvestIDGenerator(org.apache.avalon.framework.configuration.Configuration configuration)` Configures the id generator for harvests
`protected void` | `configurePipeline(org.apache.avalon.framework.configuration.Configuration configuration)` Configures the preIndexation pipeline
`protected void` | `configureStoreRepositories(java.lang.String repoUrl, org.apache.avalon.framework.configuration.Configuration oaiRepoConf)` Configures the repositories to which data will be stored based upon their repository url
`protected void` | `configureTempDir(org.apache.avalon.framework.configuration.Configuration conf)` Configures the temporary directory Configures the temporary directory where harvested documents will be stored in sub-directories.
`protected void` | `configureUpdateTriggers(java.lang.String requestUrl, org.apache.avalon.framework.configuration.Configuration updateConf)` Configures time triggers for stored requests
`protected void` | `deleteOAIDocuments()` Delete OAI documents from the current document base.
`protected void` | `deleteTempDir()` Deletes the directory represented by the tempDir class field
`protected void` | `deleteTempDirBatch()` Deletes the directory represented by the tempDirBatch class field
`void` | `endElement(java.lang.String s, java.lang.String s1, java.lang.String s2)` Receive notification of the end of an element.
`protected void` | `endHarvest()` Ends the harvest
`protected java.lang.String` | `generateNewHarvestId()` Generates an id to associate with a harvest
`protected java.lang.String` | `getHarvesterId()` Returns an id for this harvester based upon the underlying document base id
`protected IndexParameters` | `getIndexParameters()` Builds simple index parameters for indexation of oai records into the undelryi
`protected java.lang.String` | `getIsoDate()` Get's the current date in iso8601 format
`protected java.io.File` | `getNewTempDirBatch()` Creates a new temporary directory for writing harvested records before the will be indexed
`protected void` | `handleResumptionToken()` Handles the resumption token by issuing another request based upon the request from which the resumption token was received.
`protected void` | `initTempDir()` Establishes the tempDirBatch class field
`protected boolean` | `isStartsIndexation()`
`java.util.Date` | `lastUpdated()` Retrieves the time when the harvester was last updated
`protected void` | `prepareRecordCapture()` Sets up resources to capture an oai record
`protected void` | `prepareRecordForDeletion()` Sets up resources to delete an oai record Add the record to the list of the records to removed
`protected void` | `prepareResourceFromUrlIdentifierCapture()` Prepares to read a url value from an oai record and retrieve the XML behind.
`void` | `purgePastHarvestsData()` Destroys all summary data pertaining to past harvests but not the actual oai records harvested
`protected void` | `resetAllFields()` Resets necessary class fields
`protected void` | `resetRecordCaptureFields(boolean deleteDoc)` Resets the class fields for record capture possibility deleting the current `harvetDoc` object underlying file
`void` | `restore(SaveParameters save_config)` Restore the timeStamp of the Harvester
`protected void` | `saveCriticalFields(boolean dataHarvested)` Saves critical data about a harvest
`void` | `sendPastHarvestsSummary()` Sends sax events to the current consumer with summary details of the all the past harvests
`void` | `sendStoredHarvestingRequests()` Sends the details of stored harvesting requests to the current consumer
`protected boolean` | `shouldHarvestDocument()` Querys the underlying data structures based upon current sax flow position/set class fields and determines whether an oai record should be harvested
`void` | `startElement(java.lang.String s, java.lang.String s1, java.lang.String s2, org.xml.sax.Attributes attributes)` Receive notification of the beginning of an element.
`protected void` | `storeFailedHarvestData(java.lang.Exception e)` Stores data about harvesting failures caused by problems other than oai errors sent from a queried repository
`protected boolean` | `storeHarvestedData()` Reads the documents from `tempDirBatch` and indexes them in the corresponding document base, any marked deletions will be carried out as well
`void` | `targetTriggered(java.lang.String triggerName)` Triggers an OAI request to a repository based upon a trigger name (also a request url)

Methods inherited from class fr.gouv.culture.oai.AbstractOAIHarvester
`abortRecordCapture, characters, getAdminEmails, getHarvestParameters, handleErrors, receiveRequest, receiveSynchronizedRequest, receiveSynchronizedRequest, recycle, resetResumptionToken, service, setAdminEmails, setConsumer, setIdentifierName, toSAX`

Methods inherited from class fr.gouv.culture.oai.SynchronizedOAIObjectImpl
`contextualize, enableLogging, getContext, sendElement, sendElementContent`

Methods inherited from class fr.gouv.culture.util.apache.cocoon.xml.AbstractSynchronizedXMLPipe
`acquireSynchronizedXMLConsumer, comment, endCDATA, endDocument, endDTD, endEntity, endPrefixMapping, ignorableWhitespace, processingInstruction, releaseSynchronizedXMLConsumer, setDocumentLocator, skippedEntity, startCDATA, startDocument, startDTD, startEntity, startPrefixMapping`

Methods inherited from class fr.gouv.culture.util.apache.cocoon.xml.AbstractSynchronizedXMLProducer
`setConsumer`

Methods inherited from class fr.gouv.culture.util.apache.avalon.excalibur.concurrent.Mutex
`acquired, isAcquired`

Methods inherited from class fr.gouv.culture.util.apache.avalon.excalibur.concurrent.Semaphore
`acquire, attempt, getTokens, release`

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Methods inherited from interface fr.gouv.culture.oai.OAIHarvester
`getAdminEmails, receiveRequest, receiveSynchronizedRequest, receiveSynchronizedRequest, setAdminEmails, setIdentifierName`

Methods inherited from interface org.apache.avalon.framework.logger.LogEnabled
`enableLogging`

Methods inherited from interface org.apache.avalon.framework.context.Contextualizable
`contextualize`

Methods inherited from interface org.apache.excalibur.xml.sax.XMLizable
`toSAX`

Methods inherited from interface org.xml.sax.ContentHandler
`characters, endDocument, endPrefixMapping, ignorableWhitespace, processingInstruction, setDocumentLocator, skippedEntity, startDocument, startPrefixMapping`

Methods inherited from interface org.xml.sax.ext.LexicalHandler
`comment, endCDATA, endDTD, endEntity, startCDATA, startDTD, startEntity`

Methods inherited from interface fr.gouv.culture.util.apache.cocoon.xml.SynchronizedXMLProducer
`acquired, setConsumer`

Methods inherited from interface org.apache.cocoon.xml.XMLProducer
`setConsumer`

Methods inherited from interface fr.gouv.culture.util.apache.cocoon.xml.SynchronizedXMLConsumer
`acquired`

Methods inherited from interface EDU.oswego.cs.dl.util.concurrent.Sync
`acquire, attempt, release`

Field Detail

### docbase

    protected DocumentBase docbase

  The underlying document base

### docbaseId

    protected java.lang.String docbaseId

  Id of the underlying document base

### pipe

    protected Pipeline pipe

  Pre-indexation pipeline

### _database

    protected Database _database

  Underlying database to store any info

### storedRequests

    protected java.util.Hashtable storedRequests

  Requests in application.xconf

### storeRepositoriesRefs

    protected java.util.Hashtable storeRepositoriesRefs

  References to the underlying documentbase's/application's repositories

### scheduler

    protected TimeScheduler scheduler

  Time scheduler for stored requests

### harvesterIdGen

    protected IDGenerator harvesterIdGen

  IDGenerator for this object

### TEMPFILE_SUFFIX

    protected java.lang.String TEMPFILE_SUFFIX

### tempDir

    protected java.io.File tempDir

### tempDirBatch

    protected java.io.File tempDirBatch

### harvestDoc

    protected java.io.File harvestDoc

### fileOs

    protected java.io.FileOutputStream fileOs

### urlResource

    protected XMLDocument urlResource

### deletedDocs

    protected java.util.ArrayList deletedDocs

### noHarvestedDocs

    protected int noHarvestedDocs

### noDocsDeleted

    protected int noDocsDeleted

### m_docsaddedIds

    protected java.util.Set m_docsaddedIds

### m_docsToDeleteIds

    protected java.util.Set m_docsToDeleteIds

### m_docsdeletedids

    protected java.util.Set m_docsdeletedids

### keepDeletedRecords

    protected boolean keepDeletedRecords

### noRecordsPerBatch

    protected int noRecordsPerBatch

### keepHarvestedRecords

    protected boolean keepHarvestedRecords

  Force harvester to keep harvested records (default: false)

  Force harvester to keep harvested records (XML files) in file system server.
  Default is false.
  This cas be change in document base configuration file: <oai-harvester keepHarvestedRecords="{true|false}" [...]>

### tempDirPath

    protected java.lang.String tempDirPath

  Directory to store harvested documents

  Temporary path of the directory where the harvested documents will be stored.
  Default is the servlet context temp dir (eg, $TOMCAT/work/...). If the directory is not writable, the harvester will use the temporary directory of the JVM (ie, `java.io.tmpdir` system property).
  This can be change in document base configuration file:

       <oai-harvester tempDirPath="{/path/to/directory}" [...]>

  To resolve the path, harvester uses the `Utilities.resolveFile(org.apache.avalon.framework.logger.Logger, String, Context, String, boolean)`.
  By default, this directory is deleted after the harvest. This can be change with keepHarvestedRecords configuration attribute.

  See Also:
    `Utilities.resolveFile(org.apache.avalon.framework.logger.Logger, String, Context, String, boolean)`

### transformerFactory

    protected java.lang.String transformerFactory

  XML Transformer factory classe name.

  Default: Xalan, "org.apache.xalan.processor.TransformerFactoryImpl". This cas be change in configuration file: <oai-harvester transformer-factory="{classe name}" [...]>

### defaultTransformerFactory

    protected java.lang.String defaultTransformerFactory

### transformerIndent

    protected java.lang.String transformerIndent

  XML Transformer indent option.

  Default:no. This can be change in configuration file: <oai-harvester transformer-indent="yes|no" [...]>

### defaultTransformerIndent

    protected java.lang.String defaultTransformerIndent

### indexAtHarvestEnd

    protected boolean indexAtHarvestEnd

  Indexation at the end of harvesting option

  Default: true. This cas be change in configuraiton file: <oai-harvester index-at-index-end="yes|no" [...]>

### forceIndexOnHarvestError

    protected boolean forceIndexOnHarvestError

  Force indexation on harvest error option

  Default: false. This cas be change in configuraiton file: <oai-harvester force-index-on-harvest-error="yes|no" [...]>

### TRANSFORMER_FACTORY

    protected static final java.lang.String TRANSFORMER_FACTORY

See Also:
  Constant Field Values

### TRANSFORMER_INDENT

    protected static final java.lang.String TRANSFORMER_INDENT

See Also:
  Constant Field Values

### INDEXATHARVESTEND

    protected static final java.lang.String INDEXATHARVESTEND

See Also:
  Constant Field Values

### FORCEINDEXONHARVESTERROR

    protected static final java.lang.String FORCEINDEXONHARVESTERROR

See Also:
  Constant Field Values

### OAI_HARVEST_ID

    protected static final java.lang.String OAI_HARVEST_ID

See Also:
  Constant Field Values

### OAI_FAILED_HARVEST

    protected static final java.lang.String OAI_FAILED_HARVEST

See Also:
  Constant Field Values

### OAI_HARVESTER_LAST_UPDATED

    protected static final java.lang.String OAI_HARVESTER_LAST_UPDATED

See Also:
  Constant Field Values

### OAI_HARVESTER_RESUMPTION_TOKEN

    protected static final java.lang.String OAI_HARVESTER_RESUMPTION_TOKEN

See Also:
  Constant Field Values

### OAI_VERB

    protected static final java.lang.String OAI_VERB

See Also:
  Constant Field Values

### OAI_IDENTIFIER

    protected static final java.lang.String OAI_IDENTIFIER

See Also:
  Constant Field Values

### OAI_METADATA_PREFIX

    protected static final java.lang.String OAI_METADATA_PREFIX

See Also:
  Constant Field Values

### OAI_FROM

    protected static final java.lang.String OAI_FROM

See Also:
  Constant Field Values

### OAI_UNTIL

    protected static final java.lang.String OAI_UNTIL

See Also:
  Constant Field Values

### OAI_SET

    protected static final java.lang.String OAI_SET

See Also:
  Constant Field Values

### NO_DOCS_DELETED

    protected static final java.lang.String NO_DOCS_DELETED

See Also:
  Constant Field Values

### NO_DOCS_HARVESTED

    protected static final java.lang.String NO_DOCS_HARVESTED

See Also:
  Constant Field Values

### ERROR_CODE

    protected static final java.lang.String ERROR_CODE

See Also:
  Constant Field Values

### filesProperties

    protected java.util.Hashtable filesProperties

  List OAI files with OAI properties

### cBytes

    protected org.apache.cocoon.serialization.XMLSerializer cBytes

### oaiStripper

    protected org.apache.cocoon.xml.XMLPipe oaiStripper

Constructor Detail

### AbstractDocumentBaseOAIHarvester

    public AbstractDocumentBaseOAIHarvester(DocumentBase base)

  Basic constructor

Method Detail

### configure

    public void configure(org.apache.avalon.framework.configuration.Configuration configuration)
                   throws org.apache.avalon.framework.configuration.ConfigurationException

  OAI harvester configuration

  Configures the OAI harvester reading `application.xconf` file wich may contains a section such as:

       <sdx:documentBase [...]>
         <sdx:oai-harvester
                 adminEmail="{some.body@some.where}"
                 keepDeletedRecords="{true|false}"
                 noRecordsPerBatch="{number}"
                 transformer-factory="{Transformer factory classe name}"
                 transformer-indent="{yes|no}"
                 keepHarvestedRecords="{true|false}"
                 tempDirPath="{directory path}">
           <sdx:oai-data-providers>
             <sdx:oai-repository [...]>[...]</sdx:oai-repository>
             [...]
           </sdx:oai-data-providers>
         </sdx:oai-harvester>
       </sdx:documentBase>

  Specified by:
    `configure` in interface `org.apache.avalon.framework.configuration.Configurable`
  Parameters:
    `Configuration` -
  Throws:
    `org.apache.avalon.framework.configuration.ConfigurationException`
  See Also:
    `keepDeletedRecords`, `noRecordsPerBatch`, `transformerFactory`, `transformerIndent`, `keepHarvestedRecords`, `tempDirPath`

### configureTempDir

    protected void configureTempDir(org.apache.avalon.framework.configuration.Configuration conf)
                             throws org.apache.avalon.framework.configuration.ConfigurationException

  Configures the temporary directory

  Configures the temporary directory where harvested documents will be stored in sub-directories. There will be one sub-directory per batch of the harvest. This directory will be deleted after harvest. This can be change with `keepHarvestedRecords` configuration attribute.

  Parameters:
    `Configuration` -
  Throws:
    `org.apache.avalon.framework.configuration.ConfigurationException`
  See Also:
    `keepHarvestedRecords`, `tempDirPath`

### configureDatabase

    protected void configureDatabase(org.apache.avalon.framework.configuration.Configuration configuration)
                              throws org.apache.avalon.framework.configuration.ConfigurationException

  Configures the internal database

  Throws:
    `org.apache.avalon.framework.configuration.ConfigurationException`

### configureHarvestIDGenerator

    protected void configureHarvestIDGenerator(org.apache.avalon.framework.configuration.Configuration configuration)
                                        throws org.apache.avalon.framework.configuration.ConfigurationException

  Configures the id generator for harvests

  Throws:
    `org.apache.avalon.framework.configuration.ConfigurationException`

### getHarvesterId

    protected java.lang.String getHarvesterId()

  Returns an id for this harvester based upon the underlying document base id

### configureAdminEmails

    protected void configureAdminEmails(org.apache.avalon.framework.configuration.Configuration configuration)
                                 throws org.apache.avalon.framework.configuration.ConfigurationException

  Configures a list of admin emails can be sub-elements, a single attribute, or both

  Parameters:
    `configuration` -
  Throws:
    `org.apache.avalon.framework.configuration.ConfigurationException`

### configureDataProviders

    protected void configureDataProviders(org.apache.avalon.framework.configuration.Configuration configuration)
                                   throws org.apache.avalon.framework.configuration.ConfigurationException

  Configures data providers info that can be reused and from which requests can be automatically executed

  Parameters:
    `configuration` -
  Throws:
    `org.apache.avalon.framework.configuration.ConfigurationException`
  See Also:
    `storedRequests`

### configureUpdateTriggers

    protected void configureUpdateTriggers(java.lang.String requestUrl,
                                           org.apache.avalon.framework.configuration.Configuration updateConf)
                                    throws org.apache.avalon.framework.configuration.ConfigurationException

  Configures time triggers for stored requests

  Parameters:
    `requestUrl` - The request url
    `updateConf` - The configuration for updates
  Throws:
    `org.apache.avalon.framework.configuration.ConfigurationException`
  See Also:
    `scheduler`, `storedRequests`

### configureStoreRepositories

    protected void configureStoreRepositories(java.lang.String repoUrl,
                                              org.apache.avalon.framework.configuration.Configuration oaiRepoConf)
                                       throws org.apache.avalon.framework.configuration.ConfigurationException

  Configures the repositories to which data will be stored based upon their repository url

  Parameters:
    `repoUrl` - The repository/data provider url
    `oaiRepoConf` - The configuration
  Throws:
    `org.apache.avalon.framework.configuration.ConfigurationException`

### checkGranularity

    public boolean checkGranularity(java.lang.String granularity)
                             throws org.apache.avalon.framework.configuration.ConfigurationException

  Check the granularity of an AOI provider : YYYY-MM-DDThh:mm:ssZ or YYYY-MM-DD

  Parameters:
    `granularity` -
  Returns:
    true or false
  Throws:
    `org.apache.avalon.framework.configuration.ConfigurationException`

### configurePipeline

    protected void configurePipeline(org.apache.avalon.framework.configuration.Configuration configuration)
                              throws org.apache.avalon.framework.configuration.ConfigurationException

  Configures the preIndexation pipeline

  Parameters:
    `configuration` -
  Throws:
    `org.apache.avalon.framework.configuration.ConfigurationException`
  See Also:
    `pipe`

### getNewTempDirBatch

    protected java.io.File getNewTempDirBatch()
                                       throws SDXException,
                                              java.io.IOException

  Creates a new temporary directory for writing harvested records before the will be indexed

  Returns:
    File
  Throws:
    `SDXException`
    `java.io.IOException`

### deleteTempDirBatch

    protected void deleteTempDirBatch()

  Deletes the directory represented by the tempDirBatch class field

### deleteTempDir

    protected void deleteTempDir()

  Deletes the directory represented by the tempDir class field

### initTempDir

    protected void initTempDir()
                        throws SDXException,
                               java.io.IOException

  Establishes the tempDirBatch class field

  Throws:
    `SDXException`
    `java.io.IOException`

### getIsoDate

    protected java.lang.String getIsoDate()

  Get's the current date in iso8601 format

  Returns:
    String

### prepareRecordCapture

    protected void prepareRecordCapture()
                                 throws org.xml.sax.SAXException

  Sets up resources to capture an oai record

  Specified by:
    `prepareRecordCapture` in class `AbstractOAIHarvester`
  Throws:
    `org.xml.sax.SAXException`

### captureRecord

    protected void captureRecord()
                          throws java.lang.Exception

  Ends the capture of an oai record. Store properties (dateStamp and OAI identifier) needed for indexation.

  Specified by:
    `captureRecord` in class `AbstractOAIHarvester`
  Throws:
    `java.lang.Exception`

### resetRecordCaptureFields

    protected void resetRecordCaptureFields(boolean deleteDoc)

  Resets the class fields for record capture possibility deleting the current `harvetDoc` object underlying file

  Specified by:
    `resetRecordCaptureFields` in class `AbstractOAIHarvester`
  Parameters:
    `deleteDoc` - flag for deletion of actual file

### prepareRecordForDeletion

    protected void prepareRecordForDeletion()

  Sets up resources to delete an oai record Add the record to the list of the records to removed

  Specified by:
    `prepareRecordForDeletion` in class `AbstractOAIHarvester`

### isStartsIndexation

    protected boolean isStartsIndexation()

### storeHarvestedData

    protected boolean storeHarvestedData()
                                  throws org.apache.cocoon.ProcessingException,
                                         java.io.IOException,
                                         SDXException,
                                         org.xml.sax.SAXException

  Reads the documents from `tempDirBatch` and indexes them in the corresponding document base, any marked deletions will be carried out as well

  Specified by:
    `storeHarvestedData` in class `AbstractOAIHarvester`
  Returns:
    boolean
  Throws:
    `SDXException`
    `org.xml.sax.SAXException`
    `org.apache.cocoon.ProcessingException`
    `java.io.IOException`
  See Also:
    `AbstractOAIHarvester.storeHarvestedData()`

### deleteOAIDocuments

    protected void deleteOAIDocuments()
                               throws java.io.IOException,
                                      org.apache.cocoon.ProcessingException,
                                      SDXException,
                                      org.xml.sax.SAXException

  Delete OAI documents from the current document base.

  Throws:
    `java.io.IOException`
    `org.apache.cocoon.ProcessingException`
    `SDXException`
    `org.xml.sax.SAXException`

### handleResumptionToken

    protected void handleResumptionToken()

  Handles the resumption token by issuing another request based upon the request from which the resumption token was received.

  Specified by:
    `handleResumptionToken` in class `AbstractOAIHarvester`

### prepareResourceFromUrlIdentifierCapture

    protected void prepareResourceFromUrlIdentifierCapture()

  Prepares to read a url value from an oai record and retrieve the XML behind.

  Specified by:
    `prepareResourceFromUrlIdentifierCapture` in class `AbstractOAIHarvester`
  See Also:
    `AbstractOAIHarvester.identifierName`, `AbstractOAIHarvester.currentMetadtaUrlIdentifier`

### captureResourceFromUrlIdentifier

    protected void captureResourceFromUrlIdentifier()

  Captures the xml from a url taken from an oai record and adds it to the oai-record as a sibling of the element

  Specified by:
    `captureResourceFromUrlIdentifier` in class `AbstractOAIHarvester`
  See Also:
    `AbstractOAIHarvester.currentMetadtaUrlIdentifier`, `AbstractOAIHarvester.identifierName`

### resetAllFields

    protected void resetAllFields()

  Resets necessary class fields

  Overrides:
    `resetAllFields` in class `AbstractOAIHarvester`

### endHarvest

    protected void endHarvest()

  Ends the harvest

### getIndexParameters

    protected IndexParameters getIndexParameters()

  Builds simple index parameters for indexation of oai records into the undelryi

  Returns:
    IndexParameters

### sendStoredHarvestingRequests

    public void sendStoredHarvestingRequests()
                                      throws org.xml.sax.SAXException

  Sends the details of stored harvesting requests to the current consumer

  Specified by:
    `sendStoredHarvestingRequests` in interface `OAIHarvester`
  Throws:
    `org.xml.sax.SAXException`

### targetTriggered

    public void targetTriggered(java.lang.String triggerName)

  Triggers an OAI request to a repository based upon a trigger name (also a request url)

  Specified by:
    `targetTriggered` in interface `Target`
  Parameters:
    `triggerName` -

### startElement

    public void startElement(java.lang.String s,
                             java.lang.String s1,
                             java.lang.String s2,
                             org.xml.sax.Attributes attributes)
                      throws org.xml.sax.SAXException

  Description copied from class: `AbstractSynchronizedXMLPipe`
  Receive notification of the beginning of an element.

  Specified by:
    `startElement` in interface `org.xml.sax.ContentHandler`
  Overrides:
    `startElement` in class `AbstractOAIHarvester`
  Parameters:
    `s` - The Namespace URI, or the empty string if the element has no Namespace URI or if Namespace processing is not being performed.
    `s1` - The local name (without prefix), or the empty string if Namespace processing is not being performed.
    `s2` - The raw XML 1.0 name (with prefix), or the empty string if raw names are not available.
    `attributes` - The attributes attached to the element. If there are no attributes, it shall be an empty Attributes object.
  Throws:
    `org.xml.sax.SAXException`

### endElement

    public void endElement(java.lang.String s,
                           java.lang.String s1,
                           java.lang.String s2)
                    throws org.xml.sax.SAXException

  Description copied from class: `AbstractSynchronizedXMLPipe`
  Receive notification of the end of an element.

  Specified by:
    `endElement` in interface `org.xml.sax.ContentHandler`
  Overrides:
    `endElement` in class `AbstractOAIHarvester`
  Parameters:
    `s` - The Namespace URI, or the empty string if the element has no Namespace URI or if Namespace processing is not being performed.
    `s1` - The local name (without prefix), or the empty string if Namespace processing is not being performed.
    `s2` - The raw XML 1.0 name (with prefix), or the empty string if raw names are not available.
  Throws:
    `org.xml.sax.SAXException`

### shouldHarvestDocument

    protected boolean shouldHarvestDocument()

  Querys the underlying data structures based upon current sax flow position/set class fields and determines whether an oai record should be harvested

  Specified by:
    `shouldHarvestDocument` in class `AbstractOAIHarvester`
  Returns:
    boolean indicates whether the record should be handled

### saveCriticalFields

    protected void saveCriticalFields(boolean dataHarvested)
                               throws org.xml.sax.SAXException

  Saves critical data about a harvest

  Specified by:
    `saveCriticalFields` in class `AbstractOAIHarvester`
  Parameters:
    `dataHarvested` -
  Throws:
    `org.xml.sax.SAXException`

### generateNewHarvestId

    protected java.lang.String generateNewHarvestId()

  Generates an id to associate with a harvest

  Returns:
    String

### sendPastHarvestsSummary

    public void sendPastHarvestsSummary()
                                 throws org.xml.sax.SAXException

  Sends sax events to the current consumer with summary details of the all the past harvests

  Specified by:
    `sendPastHarvestsSummary` in interface `OAIHarvester`
  Throws:
    `org.xml.sax.SAXException`

### lastUpdated

    public java.util.Date lastUpdated()

  Retrieves the time when the harvester was last updated

  Returns:
    Date

### purgePastHarvestsData

    public void purgePastHarvestsData()

  Destroys all summary data pertaining to past harvests but not the actual oai records harvested

  Specified by:
    `purgePastHarvestsData` in interface `OAIHarvester`

### storeFailedHarvestData

    protected void storeFailedHarvestData(java.lang.Exception e)

  Stores data about harvesting failures caused by problems other than oai errors sent from a queried repository

  Specified by:
    `storeFailedHarvestData` in class `AbstractOAIHarvester`
  Parameters:
    `e` -

### backup

    public void backup(SaveParameters save_config)
                throws SDXException

  Save the timeStamp of the Harvester

  Specified by:
    `backup` in interface `Saveable`
  Throws:
    `SDXException`
  See Also:
    `Saveable.backup(fr.gouv.culture.sdx.utils.save.SaveParameters)`

### restore

    public void restore(SaveParameters save_config)
                 throws SDXException

  Restore the timeStamp of the Harvester

  Specified by:
    `restore` in interface `Saveable`
  Throws:
    `SDXException`
  See Also:
    `Saveable.restore(fr.gouv.culture.sdx.utils.save.SaveParameters)`

### close

    public void close()

  Close OAI harvester.

  Overrides:
    `close` in class `AbstractOAIHarvester`

Overview | Package | Class | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Copyright © 2000-2010 Ministere de la culture et de la communication / AJLSM. All Rights Reserved.