Commit d047b314 authored by Benni Mack's avatar Benni Mack
Browse files

[!!!][BUGFIX] Separate sys_history from sys_log db entries

Before, the history module fetched info about "modified records" from
sys_history+the authoritive user from a coupled sys_log entry.

Info about "insert" and "delete" was fetched from sys_log solely.

However, when using a scheduled cleanup task to truncate sys_log
then all history information is useless (see bug report).

The patch introduces a new RecordHistoryStore as an abstraction
for adding history entries (currently done solely within DataHandler).

It adds some additional, necessary SQL fields to sys_history to
store all information in there and creates an update wizard
to migrate all coupled sys_history/sys_log entries to a
new sys_history entry itself.

Additionally, the whole existing "RecordHistory" class is
now only necessary for fetching the so-called ChangeLog,
for a page or a specific record, and to do rollbacks, preparing
the history records so they can be worked on.

The whole logic for fetching the GET/POST parameters is moved
into the "ElementHistoryController", everything that is only possible
via Fluid is moved from the RecordHistory object and the
ElementHistoryController into the view.

Referencing from sys_log (Log module) into sys_history is
now done the other way around, storing information about
the corresponding history entry inside sys_log.
As a side-effect, sys_log should load faster.

Abstraction basis:
- sys_history is the only source of truth about the history of a record
- sys_log contains a reference to an history entry now
(inside sys_log.log_data) to link from the backend log module
- RecordHistoryStore exists for tracking changes to records
- RecordHistory is for retrieving, compiling the history/changelog and rollbacks
- ElementHistoryController is doing PSR-7 style request/response
handling and preparing data for the view
- Fluid is handling more view functionality now, removing
the need for doing <f:format.raw> everywhere in the templates.

Sidenotes:
* Data within sys_history is now stored as JSON, not serialized anymore
* Adding/deleting was previously stored in sys_log only, is now within sys_history
* Moving records is now tracked (but not evaluated yet)
* Highlight/Snapshot functionality within the Backend Module
was removed

This functionality is built so it can also be used within Extbase
persistence and in FE in general in a future iteration.

Resolves: #55298
Resolves: #71950
Releases: master
Change-Id: I354317609099bac10c264b9932e331fa908c98be
Reviewed-on: https://review.typo3.org/53195

Reviewed-by: Andreas Fernandez's avatarAndreas Fernandez <typo3@scripting-base.de>
Tested-by: Andreas Fernandez's avatarAndreas Fernandez <typo3@scripting-base.de>
Tested-by: default avatarTYPO3com <no-reply@typo3.com>
Reviewed-by: Joerg Kummer's avatarJoerg Kummer <typo3@enobe.de>
Tested-by: Joerg Kummer's avatarJoerg Kummer <typo3@enobe.de>
Reviewed-by: Benni Mack's avatarBenni Mack <benni@typo3.org>
Tested-by: Benni Mack's avatarBenni Mack <benni@typo3.org>
parent d80bab4c
......@@ -19,33 +19,45 @@ use Psr\Http\Message\ServerRequestInterface;
use TYPO3\CMS\Backend\History\RecordHistory;
use TYPO3\CMS\Backend\Module\AbstractModule;
use TYPO3\CMS\Backend\Template\Components\ButtonBar;
use TYPO3\CMS\Backend\Template\DocumentTemplate;
use TYPO3\CMS\Backend\Utility\BackendUtility;
use TYPO3\CMS\Core\History\RecordHistoryStore;
use TYPO3\CMS\Core\Imaging\Icon;
use TYPO3\CMS\Core\Utility\DiffUtility;
use TYPO3\CMS\Core\Utility\GeneralUtility;
use TYPO3\CMS\Fluid\View\StandaloneView;
/**
* Script Class for showing the history module of TYPO3s backend
* Controller for showing the history module of TYPO3s backend
* @see \TYPO3\CMS\Backend\History\RecordHistory
*/
class ElementHistoryController extends AbstractModule
{
/**
* @var string
* @var ServerRequestInterface
*/
public $content;
protected $request;
/**
* Document template object
* @var StandaloneView
*/
protected $view;
/**
* @var RecordHistory
*/
protected $historyObject;
/**
* Display diff or not (0-no diff, 1-inline)
*
* @var DocumentTemplate
* @var int
*/
public $doc;
protected $showDiff = 1;
/**
* @var array
*/
protected $pageInfo;
protected $recordCache = [];
/**
* Constructor
......@@ -53,10 +65,7 @@ class ElementHistoryController extends AbstractModule
public function __construct()
{
parent::__construct();
$this->getLanguageService()->includeLLFile('EXT:backend/Resources/Private/Language/locallang_show_rechis.xlf');
$GLOBALS['SOBE'] = $this;
$this->init();
$this->view = $this->initializeView();
}
/**
......@@ -69,43 +78,68 @@ class ElementHistoryController extends AbstractModule
*/
public function mainAction(ServerRequestInterface $request, ResponseInterface $response)
{
$this->main();
$response->getBody()->write($this->moduleTemplate->renderContent());
return $response;
}
$this->request = $request;
$this->moduleTemplate->getDocHeaderComponent()->setMetaInformation([]);
/**
* Initialize the module output
*/
protected function init()
{
// Create internal template object
// This is ugly, we need to remove the dependency-wiring via GLOBALS['SOBE']
// In this case, RecordHistory.php depends on GLOBALS[SOBE] being set in here
$this->doc = GeneralUtility::makeInstance(DocumentTemplate::class);
}
$lastHistoryEntry = (int)($request->getParsedBody()['historyEntry'] ?: $request->getQueryParams()['historyEntry']);
$rollbackFields = $request->getParsedBody()['rollbackFields'] ?: $request->getQueryParams()['rollbackFields'];
$element = $request->getParsedBody()['element'] ?: $request->getQueryParams()['element'];
$displaySettings = $this->prepareDisplaySettings();
$this->view->assign('currentSelection', $displaySettings);
/**
* Generate module output
*/
public function main()
{
$this->content = '<h1>' . $this->getLanguageService()->getLL('title') . '</h1>';
$this->moduleTemplate->getDocHeaderComponent()->setMetaInformation([]);
$this->showDiff = (int)$displaySettings['showDiff'];
// Start history object
$historyObj = GeneralUtility::makeInstance(RecordHistory::class);
$this->historyObject = GeneralUtility::makeInstance(RecordHistory::class, $element, $rollbackFields);
$this->historyObject->setShowSubElements((int)$displaySettings['showSubElements']);
$this->historyObject->setLastHistoryEntry($lastHistoryEntry);
if ($displaySettings['maxSteps']) {
$this->historyObject->setMaxSteps((int)$displaySettings['maxSteps']);
}
// Do the actual logic now (rollback, show a diff for certain changes,
// or show the full history of a page or a specific record)
$this->historyObject->createChangeLog();
if (!empty($this->historyObject->changeLog)) {
if ($this->historyObject->shouldPerformRollback()) {
$this->historyObject->performRollback();
} elseif ($lastHistoryEntry) {
$completeDiff = $this->historyObject->createMultipleDiff();
$this->displayMultipleDiff($completeDiff);
$this->view->assign('showDifferences', true);
$this->view->assign('fullViewUrl', $this->buildUrl(['historyEntry' => '']));
}
if ($this->historyObject->getElementData()) {
$this->displayHistory($this->historyObject->changeLog);
}
}
$elementData = $this->historyObject->getElementData();
if ($elementData) {
$this->setPagePath($elementData[0], $elementData[1]);
// Get link to page history if the element history is shown
if ($elementData[0] !== 'pages') {
$this->view->assign('singleElement', true);
$parentPage = BackendUtility::getRecord($elementData[0], $elementData[1], '*', '', false);
if ($parentPage['pid'] > 0 && BackendUtility::readPageAccess($parentPage['pid'], $this->getBackendUser()->getPagePermsClause(1))) {
$this->view->assign('fullHistoryUrl', $this->buildUrl([
'element' => 'pages:' . $parentPage['pid'],
'historyEntry' => '',
'returnUrl' => GeneralUtility::getIndpEnv('REQUEST_URI')
]));
}
}
}
$elementData = GeneralUtility::trimExplode(':', $historyObj->element);
$this->setPagePath($elementData[0], $elementData[1]);
$this->view->assign('TYPO3_REQUEST_URI', GeneralUtility::getIndpEnv('TYPO3_REQUEST_URL'));
// Get content:
$this->content .= $historyObj->main();
// Setting up the buttons and markers for docheader
$this->getButtons();
// Build the <body> for the module
$this->moduleTemplate->setContent($this->content);
$this->moduleTemplate->setContent($this->view->render());
$response->getBody()->write($this->moduleTemplate->renderContent());
return $response;
}
/**
......@@ -133,8 +167,6 @@ class ElementHistoryController extends AbstractModule
/**
* Create the panel of buttons for submitting the form or otherwise perform operations.
*
* @return array All available buttons as an assoc. array
*/
protected function getButtons()
{
......@@ -156,6 +188,281 @@ class ElementHistoryController extends AbstractModule
}
}
/**
* Displays settings evaluation
*/
protected function prepareDisplaySettings()
{
// Get current selection from UC, merge data, write it back to UC
$currentSelection = is_array($this->getBackendUser()->uc['moduleData']['history'])
? $this->getBackendUser()->uc['moduleData']['history']
: ['maxSteps' => '', 'showDiff' => 1, 'showSubElements' => 1];
$currentSelectionOverride = $this->request->getParsedBody()['settings'] ? $this->request->getParsedBody()['settings'] : $this->request->getQueryParams()['settings'];
if (is_array($currentSelectionOverride) && !empty($currentSelectionOverride)) {
$currentSelection = array_merge($currentSelection, $currentSelectionOverride);
$this->getBackendUser()->uc['moduleData']['history'] = $currentSelection;
$this->getBackendUser()->writeUC($this->getBackendUser()->uc);
}
// Display selector for number of history entries
$selector['maxSteps'] = [
10 => [
'value' => 10
],
20 => [
'value' => 20
],
50 => [
'value' => 50
],
100 => [
'value' => 100
],
999 => [
'value' => 'maxSteps_all'
]
];
$selector['showDiff'] = [
0 => [
'value' => 'showDiff_no'
],
1 => [
'value' => 'showDiff_inline'
]
];
$selector['showSubElements'] = [
0 => [
'value' => 'no'
],
1 => [
'value' => 'yes'
]
];
$scriptUrl = GeneralUtility::linkThisScript();
foreach ($selector as $key => $values) {
foreach ($values as $singleKey => $singleVal) {
$selector[$key][$singleKey]['scriptUrl'] = htmlspecialchars(GeneralUtility::quoteJSvalue($scriptUrl . '&settings[' . $key . ']=' . $singleKey));
}
}
$this->view->assign('settings', $selector);
return $currentSelection;
}
/**
* Displays a diff over multiple fields including rollback links
*
* @param array $diff Difference array
*/
protected function displayMultipleDiff(array $diff)
{
// Get all array keys needed
$arrayKeys = array_merge(array_keys($diff['newData']), array_keys($diff['insertsDeletes']), array_keys($diff['oldData']));
$arrayKeys = array_unique($arrayKeys);
if (!empty($arrayKeys)) {
$lines = [];
foreach ($arrayKeys as $key) {
$singleLine = [];
$elParts = explode(':', $key);
// Turn around diff because it should be a "rollback preview"
if ((int)$diff['insertsDeletes'][$key] === 1) {
// insert
$singleLine['insertDelete'] = 'delete';
} elseif ((int)$diff['insertsDeletes'][$key] === -1) {
$singleLine['insertDelete'] = 'insert';
}
// Build up temporary diff array
// turn around diff because it should be a "rollback preview"
if ($diff['newData'][$key]) {
$tmpArr = [
'newRecord' => $diff['oldData'][$key],
'oldRecord' => $diff['newData'][$key]
];
$singleLine['differences'] = $this->renderDiff($tmpArr, $elParts[0], $elParts[1]);
}
$elParts = explode(':', $key);
$singleLine['revertRecordUrl'] = $this->buildUrl(['rollbackFields' => $key]);
$singleLine['title'] = $this->generateTitle($elParts[0], $elParts[1]);
$lines[] = $singleLine;
}
$this->view->assign('revertAllUrl', $this->buildUrl(['rollbackFields' => 'ALL']));
$this->view->assign('multipleDiff', $lines);
}
}
/**
* Shows the full change log
*
* @param array $historyEntries
*/
protected function displayHistory(array $historyEntries)
{
if (empty($historyEntries)) {
return;
}
$languageService = $this->getLanguageService();
$lines = [];
$beUserArray = BackendUtility::getUserNames();
// Traverse changeLog array:
foreach ($historyEntries as $entry) {
// Build up single line
$singleLine = [];
// Get user names
$singleLine['backendUserUid'] = $entry['userid'];
$singleLine['backendUserName'] = $entry['userid'] ? $beUserArray[$entry['userid']]['username'] : '';
// Executed by switch user
if (!empty($entry['originaluserid'])) {
$singleLine['originalBackendUserName'] = $beUserArray[$entry['originaluserid']]['username'];
}
// Diff link
$singleLine['diffUrl'] = $this->buildUrl(['historyEntry' => $entry['uid']]);
// Add time
$singleLine['time'] = BackendUtility::datetime($entry['tstamp']);
// Add age
$singleLine['age'] = BackendUtility::calcAge($GLOBALS['EXEC_TIME'] - $entry['tstamp'], $languageService->sL('LLL:EXT:lang/Resources/Private/Language/locallang_core.xlf:labels.minutesHoursDaysYears'));
$singleLine['title'] = $this->generateTitle($entry['tablename'], $entry['recuid']);
$singleLine['elementUrl'] = $this->buildUrl(['element' => $entry['tablename'] . ':' . $entry['recuid']]);
if ((int)$entry['actiontype'] === RecordHistoryStore::ACTION_MODIFY) {
// show changes
if (!$this->showDiff) {
// Display field names instead of full diff
// Re-write field names with labels
$tmpFieldList = array_keys($entry['newRecord']);
foreach ($tmpFieldList as $key => $value) {
$tmp = str_replace(':', '', $languageService->sL(BackendUtility::getItemLabel($entry['tablename'], $value)));
if ($tmp) {
$tmpFieldList[$key] = $tmp;
} else {
// remove fields if no label available
unset($tmpFieldList[$key]);
}
}
$singleLine['fieldNames'] = implode(',', $tmpFieldList);
} else {
// Display diff
$singleLine['differences'] = $this->renderDiff($entry, $entry['tablename']);
}
}
// put line together
$lines[] = $singleLine;
}
$this->view->assign('history', $lines);
}
/**
* Renders HTML table-rows with the comparison information of an sys_history entry record
*
* @param array $entry sys_history entry record.
* @param string $table The table name
* @param int $rollbackUid If set to UID of record, display rollback links
* @return array array of records
*/
protected function renderDiff($entry, $table, $rollbackUid = 0): array
{
$lines = [];
if (is_array($entry['newRecord'])) {
/* @var DiffUtility $diffUtility */
$diffUtility = GeneralUtility::makeInstance(DiffUtility::class);
$diffUtility->stripTags = false;
$fieldsToDisplay = array_keys($entry['newRecord']);
$languageService = $this->getLanguageService();
foreach ($fieldsToDisplay as $fN) {
if (is_array($GLOBALS['TCA'][$table]['columns'][$fN]) && $GLOBALS['TCA'][$table]['columns'][$fN]['config']['type'] !== 'passthrough') {
// Create diff-result:
$diffres = $diffUtility->makeDiffDisplay(
BackendUtility::getProcessedValue($table, $fN, $entry['oldRecord'][$fN], 0, true),
BackendUtility::getProcessedValue($table, $fN, $entry['newRecord'][$fN], 0, true)
);
$rollbackUrl = '';
if ($rollbackUid) {
$rollbackUrl = $this->buildUrl(['rollbackFields' => ($table . ':' . $rollbackUid . ':' . $fN)]);
}
$lines[] = [
'title' => $languageService->sL(BackendUtility::getItemLabel($table, $fN)),
'rollbackUrl' => $rollbackUrl,
'result' => str_replace('\n', PHP_EOL, str_replace('\r\n', '\n', $diffres))
];
}
}
}
return $lines;
}
/**
* Generates the URL for a link to the current page
*
* @param array $overrideParameters
* @return string
*/
protected function buildUrl($overrideParameters = []): string
{
$params = [];
// Setting default values based on GET parameters:
if ($this->historyObject->getElementData()) {
$params['element'] = $this->historyObject->getElementString();
}
$params['historyEntry'] = $this->historyObject->lastHistoryEntry;
// Merging overriding values:
$params = array_merge($params, $overrideParameters);
// Make the link:
return BackendUtility::getModuleUrl('record_history', $params);
}
/**
* Generates the title and puts the record title behind
*
* @param string $table
* @param string $uid
* @return string
*/
protected function generateTitle($table, $uid): string
{
$title = $table . ':' . $uid;
if (!empty($GLOBALS['TCA'][$table]['ctrl']['label'])) {
$record = $this->getRecord($table, $uid);
$title .= ' (' . BackendUtility::getRecordTitle($table, $record, true) . ')';
}
return $title;
}
/**
* Gets a database record (cached).
*
* @param string $table
* @param int $uid
* @return array|NULL
*/
protected function getRecord($table, $uid)
{
if (!isset($this->recordCache[$table][$uid])) {
$this->recordCache[$table][$uid] = BackendUtility::getRecord($table, $uid, '*', '', false);
}
return $this->recordCache[$table][$uid];
}
/**
* Returns a new standalone view, shorthand function
*
* @return StandaloneView
*/
protected function initializeView()
{
/** @var StandaloneView $view */
$view = GeneralUtility::makeInstance(StandaloneView::class);
$view->setLayoutRootPaths([GeneralUtility::getFileAbsFileName('EXT:backend/Resources/Private/Layouts')]);
$view->setPartialRootPaths([GeneralUtility::getFileAbsFileName('EXT:backend/Resources/Private/Partials')]);
$view->setTemplateRootPaths([GeneralUtility::getFileAbsFileName('EXT:backend/Resources/Private/Templates')]);
$view->setTemplatePathAndFilename(GeneralUtility::getFileAbsFileName('EXT:backend/Resources/Private/Templates/RecordHistory/Main.html'));
$view->getRequest()->setControllerExtensionName('Backend');
return $view;
}
/**
* Returns LanguageService
*
......
......@@ -2,11 +2,14 @@
<f:for each="{differences}" as="differencesItem" key="key">
<div class="diff-item">
<div class="diff-item-title">
<f:format.raw>{differencesItem.title}</f:format.raw>
<f:if condition="{rollbackUrl}">
<a href="{rollbackUrl}" class="btn btn-default" style="margin-right: 5px;">{f:translate(key: 'LLL:EXT:backend/Resources/Private/Language/locallang_show_rechis.xlf:revertField')}</a>
</f:if>
{differencesItem.link} {differencesItem.title}
</div>
<div class="diff-item-result">
<div class="diff-item-result"><f:spaceless>
<f:format.raw>{differencesItem.result}</f:format.raw>
</div>
</f:spaceless></div>
</div>
</f:for>
</div>
......@@ -12,47 +12,57 @@
<th>{f:translate(key: 'LLL:EXT:backend/Resources/Private/Language/locallang_show_rechis.xlf:user')}</th>
<th>{f:translate(key: 'LLL:EXT:backend/Resources/Private/Language/locallang_show_rechis.xlf:tableUid')}</th>
<th>{f:translate(key: 'LLL:EXT:backend/Resources/Private/Language/locallang_show_rechis.xlf:differences')}</th>
<th>&nbsp;</th>
</tr>
</thead>
<tbody>
<f:for each="{history}" as="historyRow" key="key">
<f:for each="{history}" as="historyRow">
<tr>
<td><span><span title="{f:translate(key: 'LLL:EXT:backend/Resources/Private/Language/locallang_show_rechis.xlf:sumUpChanges')}">
{historyRow.rollbackLink -> f:format.raw()}
<a href="{historyRow.diffUrl}"><core:icon identifier="actions-document-history-open" /></a>
</span></span></td>
<td>{historyRow.time}</td>
<td>{historyRow.age}</td>
<td>
<be:avatar backendUser="{historyRow.backendUserUid}"/>
{historyRow.backendUserName}
<f:if condition="{historyRow.backendUserUid}">
<f:then>
{historyRow.backendUserName}
</f:then>
<f:else>
{f:translate(key: 'LLL:EXT:backend/Resources/Private/Language/locallang_show_rechis.xlf:externalChange')}
</f:else>
</f:if>
<f:if condition="{historyRow.originalBackendUserName}"> ({f:translate(key: 'LLL:EXT:backend/Resources/Private/Language/locallang_show_rechis.xlf:viaUser')} {historyRow.originalBackendUserName})</f:if>
</td>
<td>
{historyRow.tableUid -> f:format.raw()}
<a href="{elementUrl}" title="{f:translate('LLL:EXT:backend/Resources/Private/Language/locallang_show_rechis.xlf:linkRecordHistory')}">{historyRow.title}</a>
</td>
<td>
<f:if condition="{historyRow.action}">
<strong>
{historyRow.action -> f:format.raw()}
</strong>
</f:if>
<f:switch expression="{historyRow.actiontype}">
<f:case value="1">
<strong>{f:translate(key: 'LLL:EXT:backend/Resources/Private/Language/locallang_show_rechis.xlf:insert')}</strong>
</f:case>
<f:case value="4">
<strong>{f:translate(key: 'LLL:EXT:backend/Resources/Private/Language/locallang_show_rechis.xlf:delete')}</strong>
</f:case>
<f:case value="5">
<strong>{f:translate(key: 'LLL:EXT:backend/Resources/Private/Language/locallang_show_rechis.xlf:insert')}</strong>
</f:case>
</f:switch>
<f:if condition="{historyRow.fieldNames}">
{historyRow.fieldNames -> f:format.raw()}
{historyRow.fieldNames}
</f:if>
<f:if condition="{historyRow.differences}">
<f:render partial="RecordHistory/Diff" arguments="{differences: historyRow.differences}"/>
</f:if>
</td>
<td>
{historyRow.markState -> f:format.raw()}
</td>
</tr>
</f:for>
</tbody>
</table>
<f:if condition="{fullViewLink}">
<f:if condition="{fullViewUrl}">
<br/>
<f:format.raw><span class="btn btn-default">{fullViewLink}</span></f:format.raw>
<a href="{fullViewUrl}" class="btn btn-default">{f:translate(key: 'LLL:EXT:backend/Resources/Private/Language/locallang_show_rechis.xlf:fullView')}</a>
</f:if>
<br/>
<br/>
......
<h2>{f:translate(key: 'LLL:EXT:backend/Resources/Private/Language/locallang_show_rechis.xlf:mergedDifferences')}</h2>
<div>
<f:if condition="{revertAllLink}">
<f:if condition="{revertAllUrl}">
<f:then>
<f:format.raw>{revertAllLink}</f:format.raw>
<a href="{revertAllUrl}" class="btn btn-default" style="margin-right: 5px;">{f:translate(key: 'LLL:EXT:backend/Resources/Private/Language/locallang_show_rechis.xlf:revertAll')}</a>
<div
style="padding-left:10px;border-left:5px solid darkgray;border-bottom:1px dotted darkgray;padding-bottom:2px;">
<f:for each="{multipleDiff}" as="historyRow" key="key">
<h3>