欢迎投稿

今日深度:

使用python向Redis批量导入数据,pythonredis导入

使用python向Redis批量导入数据,pythonredis导入


1.使用pipeline进行批量导入数据
class Redis_Handler(Handler):
	def connect(self):
		#print self.host,self.port,self.table
		self.conn = Connection(self.host,self.port,self.table)	
		
	def execute(self, action_name):
		filename = "/tmp/temp.txt"
		batch_size = 10000
		with open(filename) as file:
			try:
				count = 0
				pipeline_redis = self.conn.client.pipeline()
				for lines in file:
					(key,value) = lines.split(',')
						count = count + 1
						if len(key)>0:
							pipeline_redis.rpush(key,value.strip())
							if not count % batch_size:
								pipeline_redis.execute()
								count = 0
			
	
				#send the last batch
				pipeline_redis.execute()
			except Exception:
				print 'redis add error'


python批量插入mysql数据问题,批量提交事务不成功

conn.commit() #你少了后面的括号
 

python批量处理excel数据

是不是new_row1有问题啊,写到一个新的sheet看看。你这一边读这个表一边修改很容易行号错误的
 

www.htsjk.Com true http://www.htsjk.com/shujukunews/2418.html NewsArticle 使用python向Redis批量导入数据,pythonredis导入 1.使用pipeline进行批量导入数据 class Redis_Handler(Handler):def connect(self):#print self.host,self.port,self.tableself.conn = Connection(self.host,self.port,self.table)de...
评论暂时关闭