Documentation

Typo3 CMS Connector

For a general introduction to the connector, please refer to RheinInsights Typo3 Enterprise Search and RAG Connector.

Typo3 Configuration

Our connector brings its own REST API for crawling. You can find the package in the connector folder below additional/typo3/rheininsights_typo3_content_connector_api-<version>.zip. You can inspect the php sources as needed before installing.

In order to use our extension, do the following:

  1. Open your Typo3 administration backend

  2. Open extensions

  3. Click on upload extension

  4. And upload the zip as provided

  5. Active the extension, if needed.

  6. Also click on Maintenance and flush the caches by clicking on flush TYPO3 and PHP cache.

Crawl User

The connector needs a crawl user which

  1. has the permissions to see all relevant contents,

  2. has access to the frontend users, frontend groups and their relationships, if you plan to use secure search

  3. Administrative privileges are not needed.

  4. MFA must stay deactivated.

Content Source Configuration

The content source configuration of the connector comprises the following mandatory configuration fields.

Typo3 configuration dialog

  1. URL, which is the fully qualified domain name or host name to the Typo3 instance’s root.

  2. API path. Path of the TYPO3 extension "content_connector_api" as configured in its extension configuration, see above. The default value is /_api/connector/v1

  3. Instance identifier. Use this to distinguish users and groups in environments where you want to crawl multiple different Typo3 instances.

  4. Username: is the username of the crawl user

  5. Password: is the password for this crawl user

  6. Public keys for SSL certificates: this configuration is needed, if you run the environment with self-signed certificates, or certificates which are not known to the Java key store.
    We use a straight-forward approach to validate SSL certificates. In order to render a certificate valid, add the modulus of the public key into this text field. You can access this modulus by viewing the certificate within the browser.

Details of a Self Signed Certificate

  1. Included sites: provide site identifiers which you would like to only have crawled. If left empty, the entire instance (while respective the excluded sites below) is crawled.

  2. Excluded sites: provide site identifiers which you would not want to have crawled. If left empty, the entire instance (while respective the included sites) is crawled.

  3. Included languages. Allows for specifying language codes or ids for indexing. If left empty, all language versions (while respecting the excluded languages) are crawled.

  4. Excluded languages: provide language identifiers or ids which you would not want to have crawled. If left empty, the entire instance (while respective the included languages) is crawled.

  5. Excluded pages. Here you can specify individual pages, which you would like to exclude from crawling.

  6. Page size. This is the number of pages fetched in one API request.

  7. Response timeout (ms). Defines how long the connector until an API call is aborted and the operation be marked as failed.

  8. Connection timeout (ms). Defines how long the connector waits for a connection for an API call.

  9. Socket timeout (ms). Defines how long the connector waits for receiving all data from an API call.

  10. The general settings are described at General Crawl Settings and you can leave these with its default values.

After entering the configuration parameters, click on validate. This validates the content crawl configuration directly against the content source. If there are issues when connecting, the validator will indicate these on the page. Otherwise, you can save the configuration and continue with Content Transformation configuration.

Limitations for Incremental Crawls and Recommended Crawl Schedules

Our API offers a change log. This means that incremental crawls can detect new and changed posts, articles and pages. However, deleted articles items will not be detected in incremental crawls.

Therefore, we recommend to configure

  1. incremental crawls to run every 15-30 minutes,

  2. as well as a weekly full scan of the documents of the instance.

  3. Principal crawls can be scheduled to run once or twice a day, depending on your use case

For more information see Crawl Scheduling .