Manticore Github Issue Search Demo

Demo by Manticore Search GitHub

👀 While it's loading, feel free to check out the other repositories:

🔔 To get a notification when it's ready, leave your email here:

By entering your email, you agree to receive notifications and marketing-related emails

Filter by reset

Search in

State

Any2387
Open513
Closed1874

Advanced

Queries are slow, about 200 million data points.

This is my configuration, version 3.4.2 runs in docker.CPU 2 core, memory 2G. searchd { listen = 9312 listen = 9308:http listen = 9306:mysql41 log = /var/log/manticore/searchd.log query_log = /var/lib/manticore/data/query.log binlog_path = /var/lib/ …

donbing007 · 2020-07-20 · 16 · #346

waitingwontfix

Dont Deprecate PHP API

jobs-git · 2019-01-24 · 7 · #135

I would say you should share the specifics of your benchmarks. Maybe there is something that can be tweaked to get better performance. In my own tests (admittedly quite crude) didnt find a appracable difference for most queries. And using mysql client was …

barryhunter · 2018-12-12

FREEZE vs optimize

sanikolaev · 2024-02-16 · 0 · #1833

❓ Per-field tokenization (question for the community) ❓

sanikolaev · 2024-05-30 · 10 · #2006

the mentioned performance reduction would only apply to tables where this feature is used The performance reduction mentioned would likely apply only to tables that utilize this feature. We would do our best to maintain the current level of performance in …

sanikolaev · 2024-03-28

Manticore 2.8 is slow

I forked benchmarks to https://github.com/abhijo89-uc/benchmarks/ and updated from plain index to rt index. then perform the same test . Its really slow !! Any comments ? Google sheet is : https://docs.google.com/spreadsheets/d/ …

abhijo89-uc · 2021-06-29 · 5 · #182

Test Profile-Guided Optimization (PGO)

zamazan4ik · 2023-07-06 · 5 · #1247

I think this is the excellent task for the community to collect PGO statistics for the section about "Performance tuning"…

tomatolog · 2023-07-06

1👍1

Performance degradation in b632b/709b9

There are several performance degradations in 20230909 b632b 709b9 (compared to 20230624 75a47 35b04 ). One of them is because I limited maximum threads in all fulltext queries to the number of physical cores. My benchmarks show that most of the time …

githubmanticore · 2023-09-20 · 1 · #1443

est::size_M

For the same statement, the first query time is always very slow, but the second query time is very fast. Is there a solution?

zhangsanhuo · 2024-07-02 · 5 · #2349

you could run you queries like set profiling = 1; query; show profile; then upload profile output for both runs to check what took most of time. Thanks for your answer. first time: second time: I tried to look up the definition of "fullscan" and found …

zhangsanhuo · 2024-07-01

log optimizing steps

Related to #1103

klirichek · 2024-09-06 · 0 · #2547

Feature Request: Different Data Formats

AbstractiveNord · 2023-06-21 · 5 · #1176

Latency overhead is sharp problem on Percolate Queries tho.

AbstractiveNord · 2023-06-21

Test Profile-Guided Optimization (PGO)

Is your feature request related to a problem? Please describe. Not a problem - just an idea of how to (possibly) improve ManticoreSearch performance. Describe the solution you'd like Recently I did a lot of testing Profile-Guided Optimization (PGO) on …

zamazan4ik · 2023-07-06 · 5 · #1247

6.2.0 gives syntax error, older versions don't

cappadaan · 2024-05-23 · 33 · #1335

bugest::NO_ESTIMATErel::6.3.0

Try disabling pseudo_sharding, it worked for me. We process a lot of queries per second on a 32 cpu machine and it ended up being faster.

cappadaan · 2023-09-08

Query optimization problems.

I have an index with 200 full text fields in it. I prepared 3.8 million data, but the data is highly repetitive. It takes 250-300 milliseconds on average to run the following query. create table test (e, s0.....s199) rt_mem_limit='1024m' min_prefix_len = …

donbing007 · 2021-01-22 · 6 · #417

Upgrading manticore from 6.0.4_230314.1a3a4ea82-1.el8 to 6.2.12_230822.dc5144d35-1.el8 results in significant cpu usage increase

digirave · 2024-01-24 · 16 · #1563

est::size_S

Here's what you can do for a fair comparison: stop both searchds purge OS cache start them wait some time inspect the load and query stats. For the latter you can calculate avg, 95p and 99p respone time from the query logs. If after that you can still see …

sanikolaev · 2023-11-05

What are the factors that affect search efficiency?

Want to consult, have those factors to affect search efficiency. Because I've learned from elasticSearch that an index should not be too much larger than available memory, so response times are millisecond level. Does manticore have the same mechanism? …

donbing007 · 2020-08-14 · 4 · #376

discussion

Is there a way to apply morphology to the data field in CALL PQ?

dubadam dubadam · 2019-04-18 · 16 · #183

bug

I'm going to close this ticket. You might create another ticket to investigate PQ document matching took 10 sec. However please post queries stream as you said about 360k rows its not clear there is some query that cause slow processing or just amount of …

tomatolog · 2019-04-18

Manticore 4.0.2 slower than Manticore 3.6.3

…Hi, I have a production 3.6.3 installation and I'm evaluating to migrate to 4.0.2, before this upgrade I'm doing some stressing tests to evaluating performances. Here the steps to compare the results: I started a VM Centos 7 with Manticore 3.6.3 …

ivanghisleni · 2021-11-24 · 17 · #650

Manticore 4.0.2 slower than Manticore 3.6.3

ivanghisleni · 2021-11-24 · 17 · #650

OK, so I assume: batch size: 1 concurrency: 4 protocol: SQL Try much bigger batch size, e.g. 1000. It should increase the performance a lot.

sanikolaev · 2021-10-22

Queries on taxi dataset are slow with ps=1

Queries on partial taxi dataset (6 indexes) are almost 2x slower with pseudo_sharding enabled: mysql> set global pseudo_sharding=0; SELECT avg(total_amount) FROM taxi WHERE trip_distance = 5; show meta; Query OK, 0 rows affected (0.00 sec) …

githubmanticore · 2023-09-05 · 1 · #1316

bugrel::6.2.0

ZONESPAN speed implications

andrisi andrisi · 2022-03-29 · 3 · #737

@tomatolog and is there a known approximate % added to the processing time? For some other features it's noted. In the a I've case the slowdown is significant, like 100x - so something is probably wrong. Using the Paragraph operator with the same dataset …

andrisi andrisi · 2022-03-28

optimize grouping by time further

A query like this is now about 10 times faster than before revamp of the date/time functions, but as discussed there still seems to be room for further optimization: mysql> SELECT year(pickup_datetime) as y, count(*) trips FROM taxi GROUP BY y ORDER BY y …

sanikolaev · 2024-02-07 · 0 · #1797

est::NO_ESTIMATE

agent_retry_count does not work with ha_strategy nodeads and noerrors

airolg airolg · 2017-10-19 · 3 · #9

Any chance to look into the CPU usage in this case? 30% system time just to pass around data seems way too much. Should I open another issue?

razvanphp · 2017-10-19

Slow SELECT ... JOIN + FACET

select ... join + facet against 2 tables of 277562 and 8671 docs is too slow: snikolaev@dev2:~$ mysql -P9306 -h0 Server version: 6.2.13 8db8be098@24031215 dev (columnar 2.2.5 aa3504b@240304) (secondary 2.2.5 aa3504b@240304) (knn 2.2.5 aa3504b@240304) git …

sanikolaev · 2024-03-19 · 2 · #1950

est::size_Mfeature::join

FATAL: out of memory

donbing007 · 2020-07-02 · 35 · #361

bugwaiting

I just want to be sure that I reproduced the same issue as you have and you got a lot of RAM segments for index what got slow

tomatolog · 2020-06-29

performance degradation with wildcard queries with many matches when disk_chunks > 1

Bug Description: There is a noticeable decrease in the speed of executing wildcard queries with an index using two or more disk chunks compared to the speed of executing the same queries with the same data using one disk chunk. This issue was not observed …

AlexeyRemenyak · 2024-07-31 · 12 · #2280

bugrel::6.3.4

Reduce RAM usage of empty plain indexes

jiru jiru · 2021-03-11 · 14 · #510

bug

Thanks for the suggestions, @barryhunter. I already tried using the builtin aliases cjk and non_cjk (for ngram_chars and charset_table respectively) without luck. It does not significantly lower RAM usage. I am already using preopen_indexes = 0 . It does …

jiru jiru · 2021-03-07

Is it possible to limit the memory usage of the searchd process

I find that after batch insert data to manticore, the usage of the searchd process not free the memory? This is my config file index hacker_new { type = rt rt_mem_limit = 2048M path = /home/mcsearch/index/hacker_new rt_attr_bigint = story_id rt_attr_ …

sangensong · 2021-04-23 · 4 · #538

waiting

Crash on OPTIMIZE INDEX

drewblin · 2019-03-19 · 17 · #170

Hi drewblin, could you rebuild all indexes from scratch with using latest code? and then recheck "optimize index prod_addr_v2" ?…

surf148 surf148 · 2019-03-05

indexer so slow

I am using Manticore's plain table to rebuild indexes from PostgreSQL using range queries. Currently, it takes about 70 minutes to process approximately 11 million records, which feels too slow. Can the process be sped up, for example, by utilizing multi- …

zhl19970919 · 2023-11-28 · 0 · #1638

What are the factors that affect search efficiency?

donbing007 · 2020-08-14 · 4 · #376

discussion

Could you provide the real query for index that you uploaded at #361 that takes several seconds to finish? I'd like to investigate what took so long for such query. could you show the output of the following set profiling=1; select id from test where …

tomatolog · 2020-07-13

Move performance tests to a separate folder, remove unnecessary script

Continued from https://github.com/manticoresoftware/manticoresearch/pull/2588 Deleted one of the scripts, added arguments to the script run. Move performance tests to a separate folder, remove unnecessary script

PavelShilin89 · 2024-09-30 · 4 · #2600

Assistance Required: Exceeding max_packet_size Limit Following Node Crash

pavelnemirovsky · 2023-11-08 · 38 · #1428

waiting

you could also try to disable pseudo sharding at the searchd section as that also could utilize more cpu but use additional memory for each shard search. However it could be better to check memory consumption by queries and figure out what query uses most …

tomatolog · 2023-09-07

Slow FT search in a plain table / single-chunk RT with high-frequency words with or w/o PS

Problem FT query takes about 1.5 minutes: mysql> select highlight() from name where match ('@username Иванов ОВ'); show meta; +-----------------------------------------+ | highlight() | +-----------------------------------------+ | Джохар Ивановец | | …

sanikolaev · 2024-05-23 · 12 · #1767

bugest::NO_ESTIMATErel::6.3.0

Slow SELECT ... JOIN + FACET

sanikolaev · 2024-03-19 · 2 · #1950

est::size_Mfeature::join

Indeed it has become a little bit slower: mysql> select title, actor_ids from movies inner join movies_liked on movies_liked.movie_id = movies.id limit 0 facet actor_ids; Empty set (18.94 sec) --- 0 out of 8671 results in 9s 469ms --- …

sanikolaev · 2024-03-19

Multi Query Subtree Slower

Describe the bug Running a multi-query does not seem to trigger subtree optimization, and is often faster when using separate queries. Looking at the logs, xN is showing for multi-queries, but wall and total cpums time is higher than when run separately. …

d3mon187 · 2023-10-18 · 0 · #1483

bugest::size_L

RAM overconsumption in 4.2.0 and 4.2.1 related with docstore cache

githubmanticore · 2022-04-15 · 12 · #767

bug

➤ Ilya Kuznetsov commented: I did some quick benchmarks on different load factors. Benchmarks included inserting, searching and deleting values from a hash that is full up to its max load factor. 0.7: 0.75: +4% 0.8: +6% 0.85: +14% 0.9: +21% 0.95: +36% 0. …

githubmanticore · 2022-04-14

Slow performance SNIPPETS on agent index

…Hi, To simplify the issue, i have 2 index: .... index WebPages { type = distributed local = WebPages0 local = WebPages1 local = WebPages2 local = WebPages3 } index WebPagesLB { type = distributed agent_persistent = sphinxserver:9312:WebPages ha_strategy = …

daikoz daikoz · 2023-03-22 · 4 · #568

bug

PQ Optimization: Common Resultset

AbstractiveNord · 2024-02-27 · 14 · #1568

est::size_M

that could work, ie create some structure to speed up query processing like Luwak does however that structure needs rebuild in case user changes PQ often It also worth to make Common subtree optimization work in the PQ

tomatolog · 2023-11-06

pseudo_sharding makes SELECT by id slower

As discussed on "dev call of Jan 20 2022" I'm creating a task to investigate and see if the performance of SELECT by id with pseudo sharding can be optimized. Now it's about 17% slower. csvpipe_command script snikolaev@dev:~$ cat 10m_simple.php <?php for …

githubmanticore · 2022-04-18 · 2 · #770

bug

For the same statement, the first query time is always very slow, but the second query time is very fast. Is there a solution?

zhangsanhuo · 2024-07-02 · 5 · #2349

you could run you queries like set profiling = 1; query; show profile; then upload profile output for both runs to check what took most of time.

tomatolog · 2024-07-01