Skip to content

Conversation

@seawinde
Copy link
Contributor

What problem does this PR solve?

Fix stats unknown when calc sync mv plan statistics

For SQLs that are related to statistics, we should not collect or compute statistics. Previously this was determined by the isInternal flag, but isInternal is too broad: it covers not only statistics-related SQL but also SQL used to generate materialized view plans. Materialized view plan generation requires statistics, so we introduce a new flag isPlanWithUnKnownColumnStats to indicate connections that are used for statistics-only operations (treat column statistics as unknown).

Issue Number: close #xxx

Related PR: #36760

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@seawinde
Copy link
Contributor Author

run buildall

@seawinde seawinde force-pushed the fix_sync_mv_stats_unknow_mv_not_chosen branch from a7000d4 to 68d17f0 Compare November 26, 2025 13:15
@seawinde
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 34211 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 68d17f0ded4c5f2275f6240149230101fb83fefa, data reload: false

------ Round 1 ----------------------------------
q1	17624	5002	4902	4902
q2	2046	331	212	212
q3	10218	1268	721	721
q4	10219	872	357	357
q5	7492	2346	2332	2332
q6	180	188	143	143
q7	909	787	648	648
q8	9344	1291	1013	1013
q9	6993	5292	5300	5292
q10	6855	2199	1808	1808
q11	505	288	290	288
q12	346	359	227	227
q13	17778	3656	3008	3008
q14	232	231	208	208
q15	584	508	512	508
q16	1043	1065	966	966
q17	582	876	352	352
q18	7556	7420	7230	7230
q19	1158	935	537	537
q20	340	337	230	230
q21	3806	3166	2252	2252
q22	1063	1035	977	977
Total cold run time: 106873 ms
Total hot run time: 34211 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4960	4923	4913	4913
q2	320	388	334	334
q3	2122	2668	2295	2295
q4	1351	1763	1298	1298
q5	4240	4616	4642	4616
q6	221	181	127	127
q7	2106	1962	1746	1746
q8	2682	2550	2464	2464
q9	7524	7545	7541	7541
q10	2998	3221	2855	2855
q11	581	533	500	500
q12	656	907	668	668
q13	3441	4076	3202	3202
q14	285	309	289	289
q15	531	513	508	508
q16	1071	1134	1094	1094
q17	1190	1589	1398	1398
q18	7945	7643	7604	7604
q19	774	884	789	789
q20	2140	2106	1837	1837
q21	4775	4266	4145	4145
q22	1082	1065	987	987
Total cold run time: 52995 ms
Total hot run time: 51210 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 183979 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 68d17f0ded4c5f2275f6240149230101fb83fefa, data reload: false

query1	1039	415	404	404
query2	6590	1556	1623	1556
query3	6751	233	228	228
query4	25960	22891	22308	22308
query5	4457	629	473	473
query6	344	231	224	224
query7	4641	485	318	318
query8	299	253	256	253
query9	8739	2541	2534	2534
query10	498	376	322	322
query11	14988	15076	14605	14605
query12	193	127	115	115
query13	1661	570	451	451
query14	10602	8824	8848	8824
query15	209	200	186	186
query16	7325	679	514	514
query17	1254	755	647	647
query18	2000	433	339	339
query19	221	203	187	187
query20	132	128	121	121
query21	218	140	113	113
query22	4122	4073	3850	3850
query23	32896	31969	32020	31969
query24	8016	2439	2436	2436
query25	600	556	485	485
query26	736	283	169	169
query27	2683	501	356	356
query28	4073	2135	2133	2133
query29	779	647	519	519
query30	313	249	203	203
query31	833	717	646	646
query32	89	77	70	70
query33	563	383	339	339
query34	790	870	533	533
query35	815	853	740	740
query36	896	959	872	872
query37	123	156	86	86
query38	3334	3283	3260	3260
query39	1485	1403	1436	1403
query40	216	124	119	119
query41	67	62	61	61
query42	138	117	107	107
query43	453	466	437	437
query44	1226	772	754	754
query45	196	193	180	180
query46	879	1004	645	645
query47	1705	1730	1650	1650
query48	396	440	320	320
query49	732	514	383	383
query50	648	683	391	391
query51	3863	3949	3843	3843
query52	108	109	110	109
query53	235	259	196	196
query54	303	290	279	279
query55	94	98	93	93
query56	337	323	314	314
query57	1149	1179	1109	1109
query58	277	269	275	269
query59	2395	2541	2454	2454
query60	357	365	345	345
query61	200	161	154	154
query62	756	708	657	657
query63	235	195	192	192
query64	3083	1167	874	874
query65	4030	3962	3956	3956
query66	1053	430	317	317
query67	15314	15005	14812	14812
query68	8448	910	641	641
query69	516	341	304	304
query70	1322	1242	1230	1230
query71	486	328	317	317
query72	5831	5115	4810	4810
query73	714	562	347	347
query74	8593	8636	8654	8636
query75	3935	3313	2856	2856
query76	3696	1124	723	723
query77	817	389	326	326
query78	9536	9788	8901	8901
query79	1626	833	609	609
query80	649	568	501	501
query81	502	273	241	241
query82	451	167	136	136
query83	267	260	261	260
query84	258	115	101	101
query85	908	488	453	453
query86	342	303	300	300
query87	3588	3452	3404	3404
query88	2833	2295	2234	2234
query89	382	330	296	296
query90	1910	224	219	219
query91	179	170	141	141
query92	81	70	65	65
query93	1116	997	673	673
query94	733	459	343	343
query95	497	409	399	399
query96	505	567	290	290
query97	2936	2995	2904	2904
query98	240	224	206	206
query99	1646	1406	1302	1302
Total cold run time: 267255 ms
Total hot run time: 183979 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 27.36 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 68d17f0ded4c5f2275f6240149230101fb83fefa, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.05	0.05
query3	0.26	0.08	0.08
query4	1.61	0.11	0.11
query5	0.29	0.25	0.26
query6	1.16	0.64	0.64
query7	0.04	0.03	0.02
query8	0.05	0.05	0.04
query9	0.58	0.53	0.52
query10	0.58	0.57	0.57
query11	0.16	0.12	0.11
query12	0.15	0.12	0.12
query13	0.63	0.59	0.61
query14	1.01	1.00	1.00
query15	0.84	0.83	0.83
query16	0.38	0.39	0.38
query17	1.03	1.04	1.06
query18	0.21	0.20	0.20
query19	1.87	1.80	1.78
query20	0.02	0.01	0.01
query21	15.43	0.19	0.13
query22	5.06	0.06	0.05
query23	15.66	0.25	0.10
query24	3.19	0.86	0.45
query25	0.07	0.07	0.06
query26	0.14	0.13	0.13
query27	0.07	0.06	0.05
query28	4.89	1.14	0.93
query29	12.60	3.86	3.18
query30	0.29	0.13	0.12
query31	2.81	0.58	0.39
query32	3.23	0.54	0.47
query33	2.99	3.09	3.02
query34	15.86	5.20	4.50
query35	4.57	4.58	4.54
query36	0.67	0.50	0.49
query37	0.09	0.07	0.07
query38	0.07	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.15	0.13
query41	0.08	0.04	0.03
query42	0.05	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 99.11 s
Total hot run time: 27.36 s

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 12.50% (1/8) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

FE Regression Coverage Report

Increment line coverage 12.50% (1/8) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants