Chinaunix首页 | 论坛 | 博客
  • 博客访问: 2900557
  • 博文数量: 199
  • 博客积分: 1400
  • 博客等级: 上尉
  • 技术积分: 4126
  • 用 户 组: 普通用户
  • 注册时间: 2008-07-06 19:06
个人简介

半个PostgreSQL DBA,热衷于数据库相关的技术。我的ppt分享https://pan.baidu.com/s/1eRQsdAa https://github.com/chenhuajun https://chenhuajun.github.io

文章分类

全部博文(199)

文章存档

2020年(5)

2019年(1)

2018年(12)

2017年(23)

2016年(43)

2015年(51)

2014年(27)

2013年(21)

2011年(1)

2010年(4)

2009年(5)

2008年(6)

分类: Mysql/postgreSQL

2015-06-19 02:00:45

1. 问题

有时候查询中会带有多个字段的查询条件,但是其中任何单个字段的选择率都不高,但是多个字段组合起来却有比较好的选择率。这种场景是bitmap索引大显身手的地方,但是bitmap索引对更新性能的影响相当大,不适合OLTP场景。PG不支持bitmap索引,但是有一个临时的内存中的类似bitmap索引的东西,叫“Bitmap Index Scan”。
除了Bitmap Index Scan,多字段索引也是一种选择,但是两种方法的性能有没有差异呢?下面作个测试看看。


2. 测试环境

测试环境在一个PC的虚拟机上
宿主机
  CPU:AMD Athlon II X4 640 3.0GHz
  MEM:6G
  OS:Win7 64bit
  虚拟机所在存储:Apacer A S510S 128GB
虚拟机
  CPU:4 core
  MEM: 2G
  OS:CentOS release 6.5 (Final)
  PostgreSQL:9.3.4(shared_buffers = 128MB,其它都是默认值)

3. 测试数据

  1. postgres=# create table tb1(c1 int,c2 int);
  2. CREATE TABLE
  3. postgres=# insert into tb1 select round(random()*100),round(random()*1000) from generate_series(1,10000000);
  4. INSERT 0 10000000
  5. postgres=# select pg_size_pretty(pg_table_size('tb1'));
  6.  pg_size_pretty
  7. ----------------
  8.  346 MB
  9. (1 row)

4. 性能对比测试

4.1 全表扫描

  1. postgres=# \timing
  2. Timing is on.
  3. postgres=# select count(*) from tb1 where c1=99 and c2=999;
  4.  count
  5. -------
  6.     92
  7. (1 row)

  8. Time: 1376.393 ms

4.2 c1单索引扫描

  1. postgres=# create index tb1_idx1 on tb1(c1);
  2. CREATE INDEX
  3. Time: 65308.131 ms
  4. postgres=# select pg_size_pretty(pg_relation_size('tb1_idx1'));
  5.  pg_size_pretty
  6. ----------------
  7.  214 MB
  8. (1 row)

  9. Time: 2.659 ms
  10. postgres=# explain (analyze,buffers) select count(*) from tb1 where c1=99 and c2=999;
  11.                                                              QUERY PLAN
  12. -------------------------------------------------------------------------------------------------------------------------------------
  13.  Aggregate (cost=47675.20..47675.21 rows=1 width=0) (actual time=1267.921..1267.921 rows=1 loops=1)
  14.    Buffers: shared read=39978
  15.    -> Bitmap Heap Scan on tb1 (cost=1891.96..47674.95 rows=99 width=0) (actual time=40.726..1267.631 rows=92 loops=1)
  16.          Recheck Cond: (c1 = 99)
  17.          Rows Removed by Index Recheck: 7363157
  18.          Filter: (c2 = 999)
  19.          Rows Removed by Filter: 99794
  20.          Buffers: shared read=39978
  21.          -> Bitmap Index Scan on tb1_idx1 (cost=0.00..1891.93 rows=102333 width=0) (actual time=30.829..30.829 rows=99886 loops=1)
  22.                Index Cond: (c1 = 99)
  23.                Buffers: shared read=276
  24.  Total runtime: 1267.961 ms
  25. (12 rows)

  26. Time: 1268.916 ms

c1单索引的选择率只有1/100,对查询性能提升毫无作用。

4.3 c2单索引扫描

  1. postgres=# drop index tb1_idx1 ;
  2. DROP INDEX
  3. Time: 41.062 ms
  4. postgres=# create index tb1_idx2 on tb1(c2);
  5. CREATE INDEX
  6. Time: 50313.915 ms
  7. postgres=# select pg_size_pretty(pg_relation_size('tb1_idx2'));
  8.  pg_size_pretty
  9. ----------------
  10.  214 MB
  11. (1 row)

  12. Time: 0.915 ms
  13. postgres=# explain (analyze,buffers) select count(*) from tb1 where c1=99 and c2=999;
  14.                                                           QUERY PLAN
  15. -------------------------------------------------------------------------------------------------------------------------------
  16.  Aggregate (cost=23558.69..23558.70 rows=1 width=0) (actual time=74.215..74.215 rows=1 loops=1)
  17.    Buffers: shared hit=1948 read=6954
  18.    -> Bitmap Heap Scan on tb1 (cost=180.85..23558.45 rows=99 width=0) (actual time=5.462..73.883 rows=92 loops=1)
  19.          Recheck Cond: (c2 = 999)
  20.          Filter: (c1 = 99)
  21.          Rows Removed by Filter: 9800
  22.          Buffers: shared hit=1948 read=6954
  23.          -> Bitmap Index Scan on tb1_idx2 (cost=0.00..180.82 rows=9652 width=0) (actual time=2.414..2.414 rows=9892 loops=1)
  24.                Index Cond: (c2 = 999)
  25.                Buffers: shared read=30
  26.  Total runtime: 74.298 ms
  27. (11 rows)

  28. Time: 76.732 ms

c1单索引的选择率有1/1000,性能提升很明显。

4.4 c1+c2索引bitmap扫描


  1. postgres=# create index tb1_idx1 on tb1(c1);
  2. CREATE INDEX
  3. Time: 56792.281 ms
  4. postgres=# explain (analyze,buffers) select count(*) from tb1 where c1=99 and c2=999;
  5.                                                                 QUERY PLAN
  6.       
  7. -------------------------------------------------------------------------------------------------------------------------------------
  8. ------
  9.  Aggregate (cost=2456.74..2456.75 rows=1 width=0) (actual time=50.347..50.348 rows=1 loops=1)
  10.    Buffers: shared hit=6587 read=343
  11.    -> Bitmap Heap Scan on tb1 (cost=2073.06..2456.49 rows=99 width=0) (actual time=39.969..50.312 rows=92 loops=1)
  12.          Recheck Cond: ((c2 = 999) AND (c1 = 99))
  13.          Rows Removed by Index Recheck: 7291
  14.          Buffers: shared hit=6587 read=343
  15.          -> BitmapAnd (cost=2073.06..2073.06 rows=99 width=0) (actual time=37.864..37.864 rows=0 loops=1)
  16.                Buffers: shared hit=30 read=276
  17.                -> Bitmap Index Scan on tb1_idx2 (cost=0.00..180.82 rows=9652 width=0) (actual time=3.091..3.091 rows=9892 loops=1)
  18.                      Index Cond: (c2 = 999)
  19.                      Buffers: shared hit=30
  20.                -> Bitmap Index Scan on tb1_idx1 (cost=0.00..1891.93 rows=102333 width=0) (actual time=33.030..33.030 rows=99886 loo
  21. ps=1)
  22.                      Index Cond: (c1 = 99)
  23.                      Buffers: shared read=276
  24.  Total runtime: 50.484 ms
  25. (15 rows)

  26. Time: 52.287 ms

c1+c2索引bitmap扫描性能进一步提升。(这个性能提升也可能仅仅是因为buffers的hit命中率提升)

4.5 多字段索引扫描

  1. postgres=# create index tb1_idx3 on tb1(c1,c2);
  2. CREATE INDEX
  3. Time: 67824.333 ms
  4. postgres=# select pg_size_pretty(pg_relation_size('tb1_idx3'));
  5.  pg_size_pretty
  6. ----------------
  7.  214 MB
  8. (1 row)

  9. Time: 0.835 ms
  10. postgres=# explain (analyze,buffers) select count(*) from tb1 where c1=99 and c2=999;
  11.                                                        QUERY PLAN
  12. -------------------------------------------------------------------------------------------------------------------------
  13.  Aggregate (cost=389.13..389.14 rows=1 width=0) (actual time=0.580..0.580 rows=1 loops=1)
  14.    Buffers: shared hit=92 read=3
  15.    -> Bitmap Heap Scan on tb1 (cost=5.45..388.89 rows=99 width=0) (actual time=0.298..0.566 rows=92 loops=1)
  16.          Recheck Cond: ((c1 = 99) AND (c2 = 999))
  17.          Buffers: shared hit=92 read=3
  18.          -> Bitmap Index Scan on tb1_idx3 (cost=0.00..5.42 rows=99 width=0) (actual time=0.279..0.279 rows=92 loops=1)
  19.                Index Cond: ((c1 = 99) AND (c2 = 999))
  20.                Buffers: shared read=3
  21.  Total runtime: 0.651 ms
  22. (9 rows)

  23. Time: 3.912 ms
由此可见,本例中,多字段索引的效率相当高。


5. 其它

5.1 多字段索引的适用条件


------------------------------------------------------------------
一个多字段的 B-tree 索引可以用在包含索引字段子集的查询条件里,不过, 如果在前导字段(最左边)上有约束条件,那么效率最高。准确的规则是前导字段上的等于约束, 加上第一个没有等于约束的非等于约束字段,将用于限制所扫描的索引范围。 将检查这两个字段右边字段上的索引以减少对表的访问,但是并不减少需要扫描的索引。比如, 假如我们有一个在(a, b, c)上的索引,查询条件是WHERE a = 5 AND b >= 42 AND c < 77, 那么索引就需要先扫描所有a= 5 且b= 42 ,直到所有a = 5 的记录扫描完毕。那些c >= 77的索引条目将被忽略,但是他们仍然会被扫描。 这个索引原则上仍然会被用于那些在b和/或c上有约束, 但是在a上没有约束的查询,但是就必须扫描整个索引了。因此,在大多数这种情况下, 优化器会选择顺序扫描表,而不使用索引。
------------------------------------------------------------------


5.2 多字段索引的原理

b-tree多字段索引的原理很简单,就是把多个字段,按定义索引时的先后顺序排序,所以越靠前的字段越重要。
下面是b-tree多字段索引条目内容的一个例子。

  1. postgres=# \d tb1
  2.       Table "public.tb1"
  3.  Column | Type | Modifiers
  4. --------+---------+-----------
  5.  id | integer |
  6.  name | text |
  7. Indexes:
  8.     "tb1_idx2" btree (id, name)

  9. postgres=# insert into tb1 values(1,'aaaa');
  10. INSERT 0 1
  11. postgres=# create extension pageinspect;
  12. CREATE EXTENSION
  13. postgres=# SELECT * FROM bt_page_items('tb1_idx2', 1);
  14.  itemoffset | ctid | itemlen | nulls | vars | data
  15. ------------+-------+---------+-------+------+-------------------------------------------------
  16.           1 | (0,5) | 24 | f | t | 01 00 00 00 0b 61 61 61 61 00 00 00 00 00 00 00
  17. (1 row)

  18. postgres=# select 'a'::bytea;
  19.  bytea
  20. -------
  21.  \x61
  22. (1 row)




阅读(16220) | 评论(0) | 转发(0) |
给主人留下些什么吧!~~