Non-Excel data source

Not all configuration data is saved in Excel format. In actual projects, some complex configurations are generated by editors, which are generally saved in formats such as json or xml. luban currently supports the following data sources:

• json
• lua
• xml
• yaml
• lite. Luban's unique concise text data format without field names is suitable for expressing very complex data structures

Types used for demonstration

Take the following DemoType2 as an example to show how to fill in the data corresponding to this type in other file formats.

<bean name="DemoType2" >
  <var name="x4" type="int"/>
  <var name="x1" type="bool"/>
  <var name="x5" type="long"/>
  <var name="x6" type="float"/>
  <var name="x7" type="double"/>
  <var name="x10" type="string"/>
  <var name="x12" type="DemoType1"/>
  <var name="x13" type="DemoEnum"/>
  <var name="x14" type="DemoDynamic#sep=,"/>
  <var name="t1" type="datetime"/>
  <var name="k1" type="array,int"/>
  <var name="k2" type="list,int"/>
  <var name="k8" type="map,int,int"/>
  <var name="k9" type="(list#sep=,#index=y1),DemoE2"/>
  <var name="k15" type="(array#sep=,),DemoDynamic"/> 
</bean>

<table name="TbDataFromSingle" value="DemoType2" input="test/datas"/> 

Data directory

If table.inputFiles points to a directory, it will automatically traverse the entire directory tree, ignore files starting with the .~_ character, and input the remaining files as data files.

These data file formats correspond to the following single record format. Note that if you encounter an excel file, it still defaults to trying to read multiple records from a file.

Single record format

json format

Some special points of the json format:

• set type. Filling method is [v1,v2,...]
• map type. Since json only supports string type keys, the map format is [[k1,v1],[k2,v2]...]
• Polymorphic bean type. The $type attribute is needed to specify the specific type name

{
  "x1":true,
  "x2":3,
  "x3":128,
  "x4":1,
  "x5":11223344,
  "x6":1.2,
  "x7":1.23432,
  "x10":"hq",
  "x12": { "x1":10},
  "x13":"B",
  "x14":{"$type": "DemoD2", "x1":1, "x2":2},
  "t1":"1970-01-01 00:00:00",
  "k1":[1,2],
  "k2":[2,3],
  "k7":[2,3],
  "k8":[[2,2],[4,10]],
  "k9":[{"y1":1, "y2":true},{"y1":2, "y2":false}],
  "k15":[{"$type": "DemoD2", "x1":1, "x2":2}]
}

lua format

Lua ​​data source features:

• There is a return before the data, because lua data is loaded as a lua file, and each loaded result is read as a record
• The format of set is {v1, v2, ...}
• Unlike json, the key of lua table supports any format, so lua map can directly {[key1] = value1, [key2] = value2, ,,,}
• Polymorphic bean type. The _type_ attribute is needed to specify the specific type name

return 
{
  x1 = false,
  x2 = 2,
  x3 = 128,
  x4 = 1122,
  x5 = 112233445566,
  x6 = 1.3,
  x7 = 1122,
  x10 = "yf",
  x12 = {x1=1},
  x13 = "D",
  x14 = { _type_="DemoD2", x1 = 1, x2=3},
  t1 = "1970-01-01 00:00:00",
  k1 = {1,2},
  k2 = {2,3},
  k8 = {[2]=10,[3]=12},
  k9 = { {y1=1,y2=true}, {y1=10,y2=false} },
  k15 = { { _type_="DemoD2", x1 = 1, x2=3} },
}

xml format

<data>
  <x1>true</x1>
  <x2>4</x2>
  <x3>128</x3>
  <x4>1</x4>
  <x5>112233445566</x5>
  <x6>1.3</x6>
  <x7>1112232.43123</x7>
  <x10>yf</x10>
  <x12> <x1>1</x1> </x12>
  <x13>C</x13>
  <x14 type="DemoD2">  <x1>1</x1>  <x2>2</x2> </x14>
  <k1> <item>1</item> <item>2</item> </k1>
  <k2> <item>1</item> <item>2</item> </k2>
  <k8>
      <item> <key>2</key><value>10</value></item>
      <item> <key>3</key><value>30</value></item>
  </k8>
  <k9>
      <item> <y1>1</y1> <y2>true</y2> </item>
      <item> <y1>2</y1> <y2>false</y2> </item>
  </k9>
  <k15>
      <item type="DemoD2"> <x1>1</x1> <x2>2</x2> </item>
  </k15>
</data>

yaml format

The characteristics of yaml format are similar to json:

• set type. The filling method corresponds to the json list [v1,v2,...]

• map type. Since json only supports string type keys, the map format filling method for the kv list in json is [[k1,v1],[k2,v2]...]

• polymorphic bean type. The $type attribute is required to specify the specific type name

tip

Files with the yaml and yml suffixes will be recognized as yaml files.

x1: true
x2: 3
x3: 128
x4: 40
x5: 11223344
x6: 1.2
x7: 1.23432
x10: hq
x12:
  x1: 10
x13: B
x14:
  $type: DemoD2
  x1: 1
  x2: 2
t1: '1970-01-01 00:00:00'
k1:
- 1
- 2
k2:
- 2
- 3
k8:
- - 2
  - 2
- - 4
  - 10
k9:
- y1: 1
  y2: true
- y1: 2
  y2: false
k15:
- $type: DemoD2
  x1: 1
  x2: 2

lite format

Lite format is a text data format unique to luban. Compared with formats such as json, it does not require field names, so it is more concise.

tip

The suffix of the Lite format file is lit, such as abc.lit.

Data type	Configuration format	Supplementary description
bool	true, false, 1, 0	Case insensitive
byte	1
short	1
int	1
long	1
float	1, 1.0
double	1, 1.0
string	abc, 'abc', "abc"
datetime	1970-01-01 00:00:00
array	{1,2,3}
list	{1,2,3}
set	{1,2,3}
map	{{1,2},{3,4},{5,6}}
enum	A
bean	{1,2,3}	Start with '{', fill in each field in order, and end with '}'. If it is a nullable polymorphic or ordinary bean, you can directly use null to represent null, but not {null}. The format of non-null polymorphic bean is {polymorphic type, field 1, field 2,...}, that is, the first field is a polymorphic type

Additional notes:

• Line breaks and blank values ​​before and after fields in the configuration will be automatically removed, similar to HTML parsing rules
• Default field values ​​are not supported, and blank data will be skipped. For example, {1,,,2} is equivalent to {1,2}
• string does not need to be surrounded by ' or " like in json format. Since blank characters before and after the string are automatically removed when parsing data, if you want to include blank fields in the string, you need to add ' or ", such as ' a '. If you encounter characters like ' or " or , or { or } or \n in a string, you need to escape them with the \ character, for example, 'abc\{\}\'\"'\t\n\\.
• Containers and beans (except beans with null values) need to start with { and end with }.

The example configuration is as follows:

Single-line filling method:

{ 1122, false, 2, 128, 112233445566, 1.3, 1122, yf, {1}, D, {DemoD2,1,3}, 1970-01-01 00:00:00, {1,2}, {2,3}, {{2,10},{3,12}}, {{1,true},{2,false}}, {{DemoD2,1,3}}}

Multi-line filling method for better readability:

{
  1122,
  false,
  2,
  128,
  112233445566,
  1.3,
  1122,
  yf,
  {1},
  D,
  {DemoD2,1,3},
  1970-01-01 00:00:00,
  {1,2},
  {2,3},
  {{2,10},{3,12}},
  {{1,true},{2,false}},
  {{DemoD2,1,3}}
}

Compound file format

If a file contains multiple records, you need to add *@ before the file name, indicating that the file contains a list of records instead of a single record. Here we take the json format as an example. The inputFiles of table have the following formats:

• xxx.json, read xxx.json as a record.

• *@xxx.json, read xxx.json as a list of records.

• field@xxx.json, read the field field in xxx.json as a record. field can be a deep-level field, such as a.b.c.

• *field@xxx.json, read the field field in xxx.json as a list of records. field can be a deep level field.

Interestingly, similar to xlsx data source, it supports putting multiple tables into the same json, but it is rarely done in practice.

As shown in the following example:

• TbCompositeJsonTable1 reads a list of records from the table1 field of composite_tables.json, reads a list of records from composite_tables2.json, and reads a record from one_record.json
• TbCompositeJsonTable2 reads a list of records from the table2 field of composite_tables.json
• TbCompositeJsonTable3 reads a record from the table3 field of composite_tables.json

<bean name="CompositeJsonTable1">
<var name="id" type="int"/>
<var name="x" type="string"/>
</bean>
<bean name="CompositeJsonTable2">
<var name="id" type="int"/>
<var name="y" type="int"/>
</bean>
<bean name="CompositeJsonTable3">
<var name="a" type="int"/>
<var name="b" type="int"/>
</bean>

<table name="TbCompositeJsonTable1" value="CompositeJsonTable1" input="*table1@composite_tables.json,*@composite_tables2.json,one_record.json"/>
<table name="TbCompositeJsonTable2" value="CompositeJsonTable2" input="*table2@composite_tables.json"/>
<table name="TbCompositeJsonTable3" value="CompositeJsonTable3" mode="one" input="table3@composite_tables.json"/>