Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] JSON 比对 #1946

Open
Jick-study opened this issue Oct 23, 2023 · 10 comments
Open

[FEATURE] JSON 比对 #1946

Jick-study opened this issue Oct 23, 2023 · 10 comments
Labels
enhancement New feature or request
Milestone

Comments

@Jick-study
Copy link

请描述您的需求或者改进建议

之前看有人提过JSON比对的方法,但是后续没啥进度了,想咨询下这个超级实用的方法还会考虑增加吗?
这个链接

@Jick-study Jick-study added the enhancement New feature or request label Oct 23, 2023
@wenshao
Copy link
Member

wenshao commented Oct 23, 2023

有考虑的,但最近比较忙,争取下个月考虑这个需求

@wenshao wenshao added this to the 2.0.43 milestone Oct 23, 2023
@Jick-study
Copy link
Author

有考虑的,但最近比较忙,争取下个月考虑这个需求

好嘞,辛苦大佬~

@wenshao wenshao modified the milestones: 2.0.43, 2.0.44 Dec 3, 2023
@wenshao wenshao modified the milestones: 2.0.44, 2.0.45, 2.0.46 Dec 23, 2023
@wenshao wenshao modified the milestones: 2.0.46, 2.0.47 Jan 27, 2024
@wenshao wenshao modified the milestones: 2.0.47, 2.0.48 Feb 23, 2024
@yanchangyou
Copy link

yanchangyou commented Feb 25, 2024

我实现了一个json对比的工具,效果如图,看看是否满足需求

21021708853869_ pic

yanchangyou pushed a commit to yanchangyou/fastjson2 that referenced this issue Feb 25, 2024
[FEATURE] JSON 比对 alibaba#1946
@Jick-study
Copy link
Author

Jick-study commented Feb 26, 2024

这个需求主要用于监控数据变化~
原始诉求如下:初始JSON、修改后的JSON
希望输出:哪些字段发生了变化,并且oldValue与newValue是什么~

[
{
"field": "字段名",
"newValue": "新的值",
"originalValue": "老的值"
}
...........
]

@yanchangyou
Copy link

yanchangyou commented Feb 26, 2024

实现思路:
1,把json按照json path平铺展开
2,对平铺展开的json进行对比
3,返回对比接口

几个主要接口:
compare: 返回所有结果
diff: 返回差异结果
sum: 对差异统计汇总
支持返回json、array

比较两个json

{
    "field":123456,
    "double":123.456789,
    "text": "ab",
    "string": "abc",
    "boolean": true,
    "object":{
        "field":123456,
        "number":123,
        "double":123.456,
        "string": "abc",
        "boolean": true,
        "newField":123
    },
    "array":[
        {
            "object":{
                "number":123,
                "double":123.456,
                "string": "abc",
                "boolean": true
            }
        }
    ]
}
{
    "field": "abc",
    "text": "abc",
    "number":123,
    "string": "abc",
    "object":{
        "field": "abc",
        "number":123,
        "string": "abc",
        "boolean": true
    },
    "array":[
        {
            "object":{
                "number":123,
                "double":123.456,
                "string": "abc"
            }
        }
    ]
}

比较结果:

{
    "equal": false,
    "total":16,
    "valueEqualCount":7,
    "typeEqualCount":1,
    "diffCount":9,
    "addCount":1,
    "removeCount":5,
    "modifyCount":3,
    "result":[
        {
            "path": "field",
            "valueEqual": false,
            "typeEqual": false,
            "diffType": "MODIFY",
            "value1":123456,
            "value2": "abc"
        },
        {
            "path": "double",
            "valueEqual": false,
            "diffType": "REMOVE",
            "value1":123.456789
        },
        {
            "path": "text",
            "valueEqual": false,
            "typeEqual": true,
            "diffType": "MODIFY",
            "value1": "ab",
            "value2": "abc"
        },
        {
            "path": "string",
            "valueEqual": true
        },
        {
            "path": "boolean",
            "valueEqual": false,
            "diffType": "REMOVE",
            "value1": true
        },
        {
            "path": "object.field",
            "valueEqual": false,
            "typeEqual": false,
            "diffType": "MODIFY",
            "value1":123456,
            "value2": "abc"
        },
        {
            "path": "object.number",
            "valueEqual": true
        },
        {
            "path": "object.double",
            "valueEqual": false,
            "diffType": "REMOVE",
            "value1":123.456
        },
        {
            "path": "object.string",
            "valueEqual": true
        },
        {
            "path": "object.boolean",
            "valueEqual": true
        },
        {
            "path": "object.newField",
            "valueEqual": false,
            "diffType": "REMOVE",
            "value1":123
        },
        {
            "path": "array[0].object.number",
            "valueEqual": true
        },
        {
            "path": "array[0].object.double",
            "valueEqual": true
        },
        {
            "path": "array[0].object.string",
            "valueEqual": true
        },
        {
            "path": "array[0].object.boolean",
            "valueEqual": false,
            "diffType": "REMOVE",
            "value1": true
        },
        {
            "path": "number",
            "valueEqual": false,
            "diffType": "ADD",
            "value2":123
        }
    ]
}

@Jick-study
Copy link
Author

顺带问下大神性能如何?因为这个功能在我们系统是个高频操作~
任何数据库的修改都需要过这个比对方法。因为我们这边需要监控每张表的每行记录变化
所以频率非常高,比较关心性能~

@yanchangyou
Copy link

yanchangyou commented Mar 4, 2024

比较json1,json2

{
	"number1":15,
	"number2":11,
	"number3":12,
	"number4":13,
	"number5":14,
	"string1":"abc",
	"string2":"abc3",
	"string3":"abc2",
	"string4":"abc1",
	"string5":"abc0"
}
{
	"number1":1.23,
	"number2":12,
	"number3":10,
	"number4":1,
	"number5":1.0,
	"string1":"abc",
	"string2":"abc1",
	"string3":"abc2",
	"string4":"abc3",
	"string5":"abc"
}

耗时统计:

次数	耗时(ms)
1024	26
2048	12
4096	18
8192	34
16384	59
32768	118
65536	243
131072	593
262144	965
524288	1929
1048576	3695

@Jick-study
Copy link
Author

那速度应该可以,看看能否融入进去~

@yanchangyou
Copy link

已经作为独立的util推送了,大佬看看怎么融合 @wenshao

#2267

@wenshao wenshao modified the milestones: 2.0.48, 2.0.49 Mar 25, 2024
@wenshao wenshao modified the milestones: 2.0.49, 2.0.50 Apr 13, 2024
@wenshao wenshao modified the milestones: 2.0.50, 2.0.51 May 11, 2024
@wenshao wenshao added this to the 2.0.52 milestone May 14, 2024
@Jick-study
Copy link
Author

这个功能很实用哇~ 预计什么时候可以合并呢~

@wenshao wenshao modified the milestones: 2.0.52, 2.0.53 Jul 7, 2024
@wenshao wenshao modified the milestones: 2.0.53, 2.0.54 Sep 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants