CSV格式与tab制表符分割的格式文件相互转换,支持管道操作

Annovar的注释结果,如果输出制表符分割的VCF格式,显得混乱。如果输出为csv格式,方便windows下的用户用excel打开,但不方便数据处理,比如某一列的注释信息中包含了逗号,这种情况就要特别注意。python中有csv模块可以方便的读取csv,推荐使用。

本文写的小脚本主要处理简单的csv格式,亮点在于支持接收标准输入和标准输出,方便生物信息多命令之间通过管道进行处理。如果没有指定输入文件,则读取管道流数据,如果没有指定输出文件,则可以用管道接收数据进行下一步处理。

比如 cat xxx.csv | python convert.py | grep "xx" > result.txt
或者 python convert.py -i input.csv | grep "xx" > result.txt
或者 python convert.py -i input.csv - o result.txt
查看用法 python convert.py --help

详细代码如下

#! /usr/bin/python
# -*- coding: utf-8 -*-
__author__ = 'Jason'
# Create: 20160114
# Modified on: 20160115
# Simple python script that convert csv to tab (-c or --csv option) or convert tab to csv (-t or --tab option).
# Support stdin and stdout

from optparse import OptionParser
import sys

def getOptions():
parser = OptionParser(usage="%prog [-i INPUT] [-csv|-tab] [-o OUT]", version="%prog 1.0")
parser.add_option("-i", "--input", type="string", dest="input", action="store", help="Input file. File should be seperated by tab or comma. If not specify, use stdin", metavar="file")
parser.add_option("-o", "--output", type="string", dest="output", action="store", help="Output file path. If not specify, use stdout.", metavar="file")
parser.add_option("-t", "--tab", default=False, dest="tab", action="store_true", help="Line field is tab splitted" )
parser.add_option("-c", "--csv", default=True, dest="csv", action="store_true", help="Line field is ',' splitted. Default True.")
(options, args) = parser.parse_args()
return options

def readLine(path):
"""
Yeild a line for processing.
:param path: Input file path. If path not specified, use stdin instead.
:return: line string
"""
if path:
for line in open(path):
yield line
else:
for line in sys.stdin.readlines():
yield line

def csv2tab(line):
"""
Just simplely convert csv to tab. Do not concert csv format with complex condition
:param line:
:return:
"""
line=line.replace(',"','"').replace('",','"') 
t=line.split('"')#if , is enclosed by " , then , in odd number 
newt=[] 
if len(t) <=2 :
return line.replace(',','t') 
else:
for i,e in enumerate(t): 
if i%2==1: 
newt.append(e) 
else: 
if e=='': continue 
newt.extend(e.split(',')) 
return 't'.join(newt) 

def tab2csv(line):
temp=[]
for t in line.split('t'):
if ',' in t:
temp.append('"%s"' % t) # if comma in field, use double quotation marks to enclose comma
else:
temp.append(t)
return ','.join(temp)

options=getOptions()
if options.output:
out=open(options.output,'w')
else: out=sys.stdout # if output path not specified, use stdout instead
for line in readLine(options.input):
if options.tab:
newLine=tab2csv(line)
else:
newLine=csv2tab(line)
out.write(newLine)
out.flush()
out.close()

发表评论

电子邮件地址不会被公开。 必填项已用*标注

This site uses Akismet to reduce spam. Learn how your comment data is processed.