Packaging: Adding missing configuration fields for property editors

[NillasKA]

Closing #20801

Description

An issue was raised, that pointed out that when creating and downloading a package from the backoffice. The blocklist data type did not contain any other properties on a block, other than 2 IDs. The missing property mentioned in the issue was the label, however i found that a bunch of other properties were missing as well.

Fix was simpler than i expected, and took some digging to figure out, but turns out we were serializing correctly, but we just did not have all of the properties in the associated models.

I added the missing properties for BlockList BlockGrid SingleBlock and RichText and refactored some of the code to ensure they each get their needed properties.

JSON Of The Needed Properties

I picked up the network requests for each of these blocks, with all properties filled out, to compare and see what blocks were needed. Here are those json snippets. So that you can compare it to the changes in the PR.

BlockList

"alias": "blocks",
      "value": [
        {
          "contentElementTypeKey": "dd183f78-7d69-4eda-9b4c-a25970583a28",
          "settingsElementTypeKey": "da15dc43-43f6-45f6-bda8-1fd17a49d25c",
          "label": "Test Label",
          "editorSize": "small",
          "iconColor": "#643b3b",
          "backgroundColor": "#9b3434",
          "thumbnail": "/wwwroot/media/03qmp3fp/24-days-people-at-codegarden.jpg",
          "forceHideContentEditorInOverlay": true
        },

RichText

      "alias": "blocks",
      "value": [
        {
          "contentElementTypeKey": "f37c2c28-c8ab-48cd-ac07-b13e38bd900f",
          "label": "Code snippet label",
          "displayInline": true,
          "editorSize": "small",
          "settingsElementTypeKey": "f37c2c28-c8ab-48cd-ac07-b13e38bd900f",
          "backgroundColor": "#c18e8e",
          "iconColor": "#9a1616",
          "thumbnail": "/wwwroot/media/03qmp3fp/24-days-people-at-codegarden.jpg",
          "forceHideContentEditorInOverlay": true
        }
      ]

BlockGrid

"alias": "blocks",
      "value": [
        {
          "contentElementTypeKey": "f37c2c28-c8ab-48cd-ac07-b13e38bd900f",
          "allowAtRoot": true,
          "allowInAreas": true,
          "label": "CODE SNIPPET LABEL",
          "settingsElementTypeKey": "f37c2c28-c8ab-48cd-ac07-b13e38bd900f",
          "columnSpanOptions": [
            {
              "columnSpan": 3
            },
            {
              "columnSpan": 9
            }
          ],
          "rowMinSpan": 12,
          "rowMaxSpan": 24,
          "areaGridColumns": 12,
          "areas": [
            {
              "key": "e54e3c04-d694-4c05-9180-26628258a322",
              "alias": "area",
              "columnSpan": 6,
              "rowSpan": 1,
              "minAllowed": 1,
              "maxAllowed": 2,
              "specifiedAllowance": [
                {
                  "minAllowed": 2,
                  "maxAllowed": 4,
                  "elementTypeKey": "f37c2c28-c8ab-48cd-ac07-b13e38bd900f"
                }
              ],
              "createLabel": "are label"
            }
          ],
          "editorSize": "small",
          "inlineEditing": true,
          "hideContentEditor": true,
          "backgroundColor": "#783939",
          "iconColor": "#5d0f0f",
          "thumbnail": "/wwwroot/media/03qmp3fp/24-days-people-at-codegarden.jpg"
        }
      ]

SingleBlock

      "alias": "blocks",
      "value": [
        {
          "contentElementTypeKey": "f37c2c28-c8ab-48cd-ac07-b13e38bd900f",
          "label": "SINGLE CODE SNIPPE ROW LABEL",
          "settingsElementTypeKey": "f37c2c28-c8ab-48cd-ac07-b13e38bd900f",
          "backgroundColor": "#7c2626",
          "iconColor": "#bc4a4a",
          "editorSize": "small",
          "thumbnail": "/wwwroot/media/03qmp3fp/24-days-people-at-codegarden.jpg",
          "forceHideContentEditorInOverlay": true
        }
      ]

Package.xml

Heres the XML for the package created using the fix.

<?xml version="1.0" encoding="utf-8"?>
<umbPackage>
  <info>
    <package>
      <name>Test Package for Bugfix</name>
    </package>
  </info>
  <DocumentTypes />
  <MediaTypes />
  <Templates />
  <Stylesheets />
  <Scripts />
  <PartialViews />
  <DictionaryItems />
  <Languages />
  <DataTypes>
    <DataType Name="[BlockList] Main Content" Id="Umbraco.BlockList" EditorUiAlias="Umb.PropertyEditorUi.BlockList" Definition="b5922818-d8d8-43df-88ed-4582a24c0fa6" DatabaseType="Ntext" Configuration="{&quot;blocks&quot;:[{&quot;contentElementTypeKey&quot;:&quot;dd183f78-7d69-4eda-9b4c-a25970583a28&quot;,&quot;settingsElementTypeKey&quot;:&quot;da15dc43-43f6-45f6-bda8-1fd17a49d25c&quot;,&quot;label&quot;:&quot;Test Label&quot;,&quot;editorSize&quot;:&quot;small&quot;,&quot;iconColor&quot;:&quot;#643b3b&quot;,&quot;backgroundColor&quot;:&quot;#9b3434&quot;,&quot;thumbnail&quot;:&quot;/wwwroot/media/03qmp3fp/24-days-people-at-codegarden.jpg&quot;,&quot;forceHideContentEditorInOverlay&quot;:true},{&quot;contentElementTypeKey&quot;:&quot;e0df4794-063a-4450-8f4f-c615a5d902e2&quot;,&quot;settingsElementTypeKey&quot;:&quot;fed88ec5-c150-42af-b444-1f9ac5a100ba&quot;,&quot;label&quot;:null,&quot;editorSize&quot;:null,&quot;iconColor&quot;:null,&quot;backgroundColor&quot;:null,&quot;thumbnail&quot;:null,&quot;forceHideContentEditorInOverlay&quot;:null},{&quot;contentElementTypeKey&quot;:&quot;f43c8349-0801-44b8-9113-9f7c62cd44fe&quot;,&quot;settingsElementTypeKey&quot;:&quot;eef34ceb-ddf6-4894-b1ac-f96c8c05d3d2&quot;,&quot;label&quot;:null,&quot;editorSize&quot;:null,&quot;iconColor&quot;:null,&quot;backgroundColor&quot;:null,&quot;thumbnail&quot;:null,&quot;forceHideContentEditorInOverlay&quot;:null},{&quot;contentElementTypeKey&quot;:&quot;f37c2c28-c8ab-48cd-ac07-b13e38bd900f&quot;,&quot;settingsElementTypeKey&quot;:&quot;93638715-f76c-4a11-86b1-6a9d66504901&quot;,&quot;label&quot;:null,&quot;editorSize&quot;:null,&quot;iconColor&quot;:null,&quot;backgroundColor&quot;:null,&quot;thumbnail&quot;:null,&quot;forceHideContentEditorInOverlay&quot;:null},{&quot;contentElementTypeKey&quot;:&quot;1c43fe2d-4a9a-4336-923f-9d0214950d48&quot;,&quot;settingsElementTypeKey&quot;:&quot;378fde96-51b6-4506-93e3-ec3038e636bb&quot;,&quot;label&quot;:null,&quot;editorSize&quot;:null,&quot;iconColor&quot;:null,&quot;backgroundColor&quot;:null,&quot;thumbnail&quot;:null,&quot;forceHideContentEditorInOverlay&quot;:null},{&quot;contentElementTypeKey&quot;:&quot;60085a63-b77b-4509-9df4-bcb75db2755f&quot;,&quot;settingsElementTypeKey&quot;:&quot;c56fb5b8-0b89-4206-847e-a6fecd865b84&quot;,&quot;label&quot;:null,&quot;editorSize&quot;:null,&quot;iconColor&quot;:null,&quot;backgroundColor&quot;:null,&quot;thumbnail&quot;:null,&quot;forceHideContentEditorInOverlay&quot;:null}],&quot;validationLimit&quot;:{&quot;min&quot;:null,&quot;max&quot;:null},&quot;useSingleBlockMode&quot;:false}" />
    <DataType Name="BLOCK GRID" Id="Umbraco.BlockGrid" EditorUiAlias="Umb.PropertyEditorUi.BlockGrid" Definition="4291b33d-fc9b-4f55-9d17-b59d1bcf57e3" DatabaseType="Ntext" Configuration="{&quot;blocks&quot;:[{&quot;areas&quot;:[{&quot;key&quot;:&quot;e54e3c04-d694-4c05-9180-26628258a322&quot;,&quot;alias&quot;:&quot;area&quot;,&quot;columnSpan&quot;:6,&quot;rowSpan&quot;:1,&quot;minAllowed&quot;:1,&quot;maxAllowed&quot;:2,&quot;specifiedAllowance&quot;:[{&quot;minAllowed&quot;:2,&quot;maxAllowed&quot;:4,&quot;elementTypeKey&quot;:&quot;f37c2c28-c8ab-48cd-ac07-b13e38bd900f&quot;}],&quot;createLabel&quot;:&quot;are label&quot;}],&quot;contentElementTypeKey&quot;:&quot;f37c2c28-c8ab-48cd-ac07-b13e38bd900f&quot;,&quot;settingsElementTypeKey&quot;:&quot;f37c2c28-c8ab-48cd-ac07-b13e38bd900f&quot;,&quot;label&quot;:&quot;CODE SNIPPET LABEL&quot;,&quot;editorSize&quot;:&quot;small&quot;,&quot;iconColor&quot;:&quot;#5d0f0f&quot;,&quot;backgroundColor&quot;:&quot;#783939&quot;,&quot;thumbnail&quot;:&quot;/wwwroot/media/03qmp3fp/24-days-people-at-codegarden.jpg&quot;,&quot;forceHideContentEditorInOverlay&quot;:null,&quot;displayInline&quot;:null,&quot;allowAtRoot&quot;:true,&quot;allowInAreas&quot;:true,&quot;hideContentEditor&quot;:true,&quot;rowMinSpan&quot;:12,&quot;rowMaxSpan&quot;:24,&quot;areaGridColumns&quot;:12}],&quot;validationLimit&quot;:{&quot;min&quot;:null,&quot;max&quot;:null},&quot;gridColumns&quot;:12}" />
    <DataType Name="SINGLE BLOCK EDITOR" Id="Umbraco.SingleBlock" EditorUiAlias="Umb.PropertyEditorUi.BlockSingle" Definition="65cba517-75e8-401e-a570-786ca9f9d0dd" DatabaseType="Ntext" Configuration="{&quot;blocks&quot;:[{&quot;contentElementTypeKey&quot;:&quot;f37c2c28-c8ab-48cd-ac07-b13e38bd900f&quot;,&quot;settingsElementTypeKey&quot;:&quot;f37c2c28-c8ab-48cd-ac07-b13e38bd900f&quot;,&quot;label&quot;:&quot;SINGLE CODE SNIPPE ROW LABEL&quot;,&quot;editorSize&quot;:&quot;small&quot;,&quot;iconColor&quot;:&quot;#bc4a4a&quot;,&quot;backgroundColor&quot;:&quot;#7c2626&quot;,&quot;thumbnail&quot;:&quot;/wwwroot/media/03qmp3fp/24-days-people-at-codegarden.jpg&quot;,&quot;forceHideContentEditorInOverlay&quot;:true}]}" />
    <DataType Name="Richtext editor" Id="Umbraco.RichText" EditorUiAlias="Umb.PropertyEditorUi.Tiptap" Definition="ca90c950-0aff-4e72-b976-a30b1ac57dad" DatabaseType="Ntext" Configuration="{&quot;blocks&quot;:[{&quot;contentElementTypeKey&quot;:&quot;f37c2c28-c8ab-48cd-ac07-b13e38bd900f&quot;,&quot;settingsElementTypeKey&quot;:&quot;a0eec50c-54ce-47d3-97f1-c01843887567&quot;,&quot;label&quot;:&quot;123&quot;,&quot;editorSize&quot;:&quot;small&quot;,&quot;iconColor&quot;:&quot;#d71616&quot;,&quot;backgroundColor&quot;:&quot;#6a2626&quot;,&quot;thumbnail&quot;:&quot;/wwwroot/media/03qmp3fp/24-days-people-at-codegarden.jpg&quot;,&quot;forceHideContentEditorInOverlay&quot;:true,&quot;displayInline&quot;:true}],&quot;mediaParentId&quot;:null,&quot;ignoreUserStartNodes&quot;:false}" />
  </DataTypes>
  <MediaItems />
</umbPackage>

[AndyButland]

@NillasKA - as we discussed on Friday with @nielslyngsoe and others, we should see if we can find a different approach here. Going this way means we need to ensure that every configuration elements created for a data type is known on the server, which isn't the model we've adopted so far in 14+. Now we allow the front-end to register it's own configuration values for data types, that purely have a presentation concern. And the data type configuration models known on the server are only those needed on the server, e.g. for server-side validation.

We could in theory do what you've started in this PR, and just make sure we do have all the core ones known on the server. But it's a maintenance burden to maintain, and we can't be sure property editors from packages will adhere to this.

I've had a look and I think we can deal with this if we work with the ConfigurationData rather than the ConfigurationObject defined on IDataType. The latter are the raw values the server stores, the former are the subset of these that have meaning on the server. So the root cause of the problem we are trying to address with this PR, is that we are using ConfigurationObject and therefore missing various presentation only configuration properties.

I've looked at amending this in EntityXmlSerializer.Serialize(IDataType dataType):

-        xml.Add(new XAttribute("Configuration", _configurationEditorJsonSerializer.Serialize(dataType.ConfigurationObject)));
+        xml.Add(new XAttribute("Configuration", SerializeDataTypeConfiguration(dataType)));

And defining this private method:

    private string SerializeDataTypeConfiguration(IDataType dataType) =>

        // We have two properties containing configuration data:
        // 1. ConfigurationData - a dictionary that contains all the configuration data stored as key/value pairs.
        // 2. ConfigurationObject - a strongly typed object that represents the configuration data known to the server.
        // To fully be able to restore the package, we need to serialize the full ConfigurationData dictionary, not
        // just the configuration properties known to the server.
        _configurationEditorJsonSerializer.Serialize(dataType.ConfigurationData);

With that in place I've then:

• Created a data type based on the block list and configured various presentation properties - e.g. colours, labels, widths.
• Created a package, added this data type to it and downloaded the package.xml file.
• Then I've deleted the data type.
• In a new project I've created an Umbraco extension that has an automatic package migration plan and the package.xml file as an embedded resource (see docs for details on this).
• From that project I've run dotnet pack and created an .nupkg file.
• I've then added that as a reference to the Umbraco.Web.UI using a local NuGet feed:
  • Copy the .nupkg file to a local folder
  • Update the NuGet.config project to add a feed from that folder, e.g.:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <clear />
    <add key="nuget" value="https://api.nuget.org/v3/index.json" />
    <add key="Local Feed" value="c:\nuget" />
  </packageSources>
  <activePackageSource>
    <add key="All" value="(Aggregate source)" />
  </activePackageSource>
</configuration>

• Then restarting the Umbraco application should run the migration registered with that package, that will then (re-)create the data type.
• I've verified that the various properties I originally configured on the data type are in place.

So I think this is probably the most fruitful approach to take here. When you are back next week, please could you see if you can try this yourself - maybe test with a few more data types with different configurations - and see if you can see any concerns with handling the issue in this way?