《Mysql應(yīng)用刪除MySQL重復(fù)數(shù)據(jù)的方法》要點(diǎn)：
本文介紹了Mysql應(yīng)用刪除MySQL重復(fù)數(shù)據(jù)的方法，希望對(duì)您有用。如果有疑問，可以聯(lián)系我們。
本文實(shí)例講述了刪除MySQL重復(fù)數(shù)據(jù)的辦法.分享給大家供大家參考.具體辦法如下：MYSQL應(yīng)用
項(xiàng)目背景MYSQL應(yīng)用
在最近做的一個(gè)linux性能采集項(xiàng)目中,發(fā)現(xiàn)線程的程序入庫很慢,再仔細(xì)定位,發(fā)現(xiàn)數(shù)據(jù)庫里面很多冗余數(shù)據(jù).因?yàn)樵诓杉?對(duì)于同一臺(tái)設(shè)備,同一個(gè)時(shí)間點(diǎn)應(yīng)該只有一個(gè)數(shù)據(jù),然而,數(shù)據(jù)庫中存入了多個(gè)數(shù)據(jù).對(duì)于如何造成了這個(gè)結(jié)果,一時(shí)沒有想清楚,但為了辦理入庫慢的問題,首先要?jiǎng)h除冗余數(shù)據(jù).MYSQL應(yīng)用
問題描述MYSQL應(yīng)用
數(shù)據(jù)庫的表結(jié)構(gòu)很簡(jiǎn)單,如下：
MYSQL應(yīng)用
代碼如下:
+----------------+--------------+------+-----+---------+-------+
| Field????????? | Type???????? | Null | Key | Default | Extra |
+----------------+--------------+------+-----+---------+-------+
| id???????????? | varchar(255) | NO?? | PRI | NULL??? |?????? |
| conf_id??????? | varchar(255) | NO?? | MUL | NULL??? |?????? |
| insert_time??? | datetime???? | YES? |???? | NULL??? |?????? |
| cpu_usage????? | float(11,2)? | YES? |???? | NULL??? |?????? |
| memory_usage?? | float(11,2)? | YES? |???? | NULL??? |?????? |
| io_usage_write | float(11,2)? | YES? |???? | NULL??? |?????? |
| io_usage_read? | float(11,2)? | YES? |???? | NULL??? |?????? |
+----------------+--------------+------+-----+---------+-------+
查詢所有數(shù)據(jù)量
代碼如下:
select count(*) from perf_linux;
輸出 427366
查詢所有時(shí)間點(diǎn)不同設(shè)備的數(shù)據(jù)量
MYSQL應(yīng)用
代碼如下:
select count(distinct conf_id, insert_time) from perf_linux ;
輸出42387
由上面的數(shù)據(jù)可以看出,數(shù)據(jù)冗余了10倍左右.MYSQL應(yīng)用
再按時(shí)間分組看一下:
MYSQL應(yīng)用
代碼如下:
select id, conf_id ,insert_time from perf_linux order by insert_time, conf_id;
輸出：
代碼如下:
| 2a79f7cd-43a9-4c7b-adb2-316b6c04283e | 1?????? | 2014-12-09 15:09:14 |
| 50d6f6c2-9c8b-45fd-98fd-2be211221cfd | 1?????? | 2014-12-09 15:09:14 |
| 740b52e1-e868-4074-ba36-74e2634401b3 | 1?????? | 2014-12-09 15:09:14 |
| 8b0096a4-9e85-417b-a131-e3505ca79a9c | 1?????? | 2014-12-09 15:09:14 |
| 90a9e882-5220-4508-a56f-8d4ab4a7929b | 1?????? | 2014-12-09 15:09:14 |
| d17403ed-24a4-45e8-b51b-2a95118383d9 | 1?????? | 2014-12-09 15:09:14 |
| 0c2da917-579b-4080-857d-7159f38b44ac | 2?????? | 2014-12-09 15:09:14 |
| 263083eb-8f63-4d2b-a03f-3320aa678735 | 2?????? | 2014-12-09 15:09:14 |
| d6c57a38-080b-465a-a55a-beafd9daf32d | 2?????? | 2014-12-09 15:09:14 |
| f672227b-1fb8-4b85-880d-2cc34b02880d | 2?????? | 2014-12-09 15:09:14 |
| f80020fe-6cb5-48ec-beb0-4e8ebeb0ca57 | 2?????? | 2014-12-09 15:09:14 |
| ff633a35-824d-49ba-b78c-5bcc5df8d1cc | 2?????? | 2014-12-09 15:09:14 |
| 5c41e48a-abfc-4108-a00e-ca7def7d5a5a | 3?????? | 2014-12-09 15:09:14 |
| 60b7ab9e-c91a-4020-a6d3-7bceb1dc47c5 | 3?????? | 2014-12-09 15:09:14 |
| 7b6cd2b8-ac6d-43eb-8858-e15885e676c8 | 3?????? | 2014-12-09 15:09:14 |
| d53a3df5-08c4-4604-8fac-cb51077935f6 | 3?????? | 2014-12-09 15:09:14 |
| d9e4ba14-f98d-42a8-b3bc-2879d58aa797 | 3?????? | 2014-12-09 15:09:14 |
| f56f82f6-32a7-47f7-ae07-b13168743884 | 3?????? | 2014-12-09 15:09:14 |
| 076c4c1b-0028-4a9c-a8c4-de655bd6ab6b | 4?????? | 2014-12-09 15:09:14 |
| 2a90ad9e-11a5-4707-95e8-78491da658ad | 4?????? | 2014-12-09 15:09:14 |
| 3b17ad1d-e589-4b65-93a7-d61fc99b4071 | 4?????? | 2014-12-09 15:09:14 |
| 6988d6cf-44ef-47f7-808d-09791caf2d90 | 4?????? | 2014-12-09 15:09:14 |
| 8404d281-f9e5-4153-a47e-128c05386758 | 4?????? | 2014-12-09 15:09:14 |
| e042e310-7ff2-4e4d-8c98-71e3e4d57828 | 4?????? | 2014-12-09 15:09:14 |
+--------------------------------------+---------+---------------------+
由上圖可見,同一個(gè)時(shí)間點(diǎn)的同一個(gè)設(shè)備的數(shù)據(jù)有冗余,現(xiàn)在我們要把這些冗余數(shù)據(jù)去掉.
解決辦法MYSQL應(yīng)用
思路是這樣的：首先應(yīng)該依照conf_id和時(shí)間點(diǎn)來判斷,進(jìn)行分組(group by)查詢,每組中再取一個(gè)就可以.分組是很簡(jiǎn)單,但是分組怎么取一個(gè)呢?我采用了中間表的形式.MYSQL應(yīng)用
創(chuàng)建中間表,并把數(shù)據(jù)導(dǎo)入中間表
MYSQL應(yīng)用
代碼如下:
create table perf_linux_t like perf_linux;
insert into perf_linux_t select * from perf_linux;
在中間表中增加一個(gè)字段,此字段是自增長(zhǎng)的.
代碼如下:
ALTER TABLE `perf_linux_t`
ADD COLUMN `auto_id` INT NOT NULL AUTO_INCREMENT ,
DROP PRIMARY KEY,
ADD PRIMARY KEY (`auto_id`);
刪除無用數(shù)據(jù)
先查詢一下
MYSQL應(yīng)用
代碼如下:
select min(auto_id) as auto_id from perf_linux_t group by insert_time ;
刪除不對(duì)的數(shù)據(jù)
代碼如下:
delete? from perf_linux_t where auto_id not in (select min(auto_id) as auto_id from perf_linux_t group by insert_time);
慢著,輸出錯(cuò)誤：
You can't specify target table 'perf_linux_t' for update in FROM clauseMYSQL應(yīng)用
不能刪除啊,那只能再建一個(gè)中間表了.MYSQL應(yīng)用
再建中間表
MYSQL應(yīng)用
代碼如下:
create table tmp like perf_linux_t;
轉(zhuǎn)變思路,不刪除不符合的數(shù)據(jù),而是把符合的數(shù)據(jù)存到這張新表中.
代碼如下:
insert into tmp select * from perf_linux_t where auto_id in (select min(auto_id) as auto_id from perf_linux_t group by insert_time,conf_id );
把這張表中的無用列刪除
代碼如下:
ALTER TABLE `tmp`
DROP COLUMN `auto_id`,
DROP PRIMARY KEY;
導(dǎo)回?cái)?shù)據(jù)
刪除原來的數(shù)據(jù)
MYSQL應(yīng)用
代碼如下:
truncate table perf_linux;
插入數(shù)據(jù)
代碼如下:
insert into perf_linux select * from tmp;
刪除中間表
代碼如下:
drop table tmp;
drop table perf_linux_t;
總結(jié)
通過這個(gè)辦法,數(shù)據(jù)變?yōu)榱?2387條,刪除了冗余的數(shù)據(jù).但實(shí)際上程序的問題并沒有完全定位,還需要觀察才能定位問題.MYSQL應(yīng)用
希望本文所述對(duì)大家的mysql數(shù)據(jù)庫程序設(shè)計(jì)有所贊助.MYSQL應(yīng)用
《Mysql應(yīng)用刪除MySQL重復(fù)數(shù)據(jù)的方法》是否對(duì)您有啟發(fā)，歡迎查看更多與《Mysql應(yīng)用刪除MySQL重復(fù)數(shù)據(jù)的方法》相關(guān)教程，學(xué)精學(xué)透。維易PHP學(xué)院為您提供精彩教程。
轉(zhuǎn)載請(qǐng)注明本頁網(wǎng)址：
http://www.fzlkiss.com/jiaocheng/12931.html
