gogetxueqiu

package module
v0.0.0-...-c30358d Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 21, 2017 License: MIT Imports: 17 Imported by: 0

README

GoGetXueQiu

A gogetxueqiu package for getting stocks and portfolios info from xueqiu.com, in Golang (Go).

本项目是用 Golang 实现的一个针对雪球的爬虫系统，模拟普通用户登录，并从雪球的 json API 获取信息。登录完成后，各接口函数支持 go 并发调用。

依赖第三方包

"github.com/bitly/go-simplejson"

使用方法

go get "github.com/ynsfsmj/gogetxueqiu"

API 用法请参考 examples/ 文件：

开发完成情况

实现了模拟用户登录（解决 CSRF 保护等问题）

初步实现了请求 API 的 Demo

接下来：

  1. 实现已知 API 的适配，解决 API 数据获取问题

  2. 利用 Go 的优势，实现并发抓取数据，提高效率，但尽量避免高频访问而被封禁

  3. 开发实现数据本地缓存，减少抓取数据频率，并提供二次 API，供大家使用

  4. 增加数据分析和展示功能

项目介绍

本项目是一个针对雪球的爬虫系统，模拟普通用户登录，并从雪球的 json API 获取信息。

在模拟用户登录后，可以访问的信息如下所示：（包括但不限）

  • 雪球组合

雪球组合净值变化（按天） https://xueqiu.com/cubes/nav_daily/all.json?cube_symbol=ZH024581&since=1453555757000&until=1461331757000

组合调仓记录 https://xueqiu.com/cubes/rebalancing/history.json?cube_symbol=ZH024581&count=20&page=1

雪球当日热门组合列表 https://xueqiu.com/cubes/discover/rank/cube/list.json?category=14

雪球组合站内排名 https://xueqiu.com/cubes/data/rank_percent.json?cube_id=24482&market=cn&dimension=annual

雪球组合评分 https://xueqiu.com/cubes/rank/summary.json?symbol=ZH024581&ua=web

雪球组合管理者时间线 https://xueqiu.com/cube/timeline?symbol=ZH024581&page=1&count=20&comment=0&uid=9188557237

  • 雪球股票

雪球股票当前状态和价格 https://xueqiu.com/v4/stock/quote.json?code=SZ000625

雪球股票盘口交易数据 https://xueqiu.com/stock/pankou.json?symbol=SZ000625

雪球股票当日分时图 https://xueqiu.com/stock/forchart/stocklist.json?symbol=SZ000625&period=1d&one_min=1

雪球股票后复权日线图 https://xueqiu.com/stock/forchartk/stocklist.json?symbol=SZ000625&period=1day&type=before&begin=1429798115327&end=1461334115327

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var XueqiuAccounts = map[string]string{}

XueqiuAccounts : xueqiu accounts

View Source 
var XueqiuUrls = map[string]string{

	"csrf": "https://xueqiu.com/service/csrf",

	"login": "https://xueqiu.com/user/login",

	"stock_rt": "https://xueqiu.com/v4/stock/quote.json",

	"stock_k_list": "https://xueqiu.com/stock/forchartk/stocklist.json",

	"stock_minutes": "https://xueqiu.com/stock/forchart/stocklist.json",

	"stock_buysell": "https://xueqiu.com/stock/pankou.json",

	"stocks_price": "https://xueqiu.com/stock/quotep.json",

	"pf_daily": "https://xueqiu.com/cubes/nav_daily/all.json",

	"pf_recommend": "https://xueqiu.com/cubes/discover/rank/cube/list.json?category=14",

	"pf_rank_percent": "https://xueqiu.com/cubes/data/rank_percent.json",

	"pf_rebalance": "https://xueqiu.com/cubes/rebalancing/history.json",

	"pf_scores": "https://xueqiu.com/cubes/rank/summary.json",
}

XueqiuUrls : xueqiu urls map

Functions

func GetMd5HexStr 

func GetMd5HexStr(str string) string

GetMd5HexStr : calculate MD5

func HTTPGet 

func HTTPGet(urlBase string, params map[string]string) (int, string, error)

HTTPGet : request using GET method, will encode the get params into url

func HTTPGetBytes 

func HTTPGetBytes(urlBase string, params map[string]string) (int, []byte, error)

HTTPGetBytes : request using GET method, will encode the get params into url

func HTTPGetJSON 

func HTTPGetJSON(urlBase string, params map[string]string) (int, *simplejson.Json, error)

HTTPGetJSON : request using GET method, will encode the get params into url

func HTTPPost 

func HTTPPost(urlBase string, params map[string]string) (int, string, error)

HTTPPost : request using POST method, will encode the post params into formBody

func Login 

func Login() (string, error)

Login : return ok, username

Types

type PfBasic 

type PfBasic struct {
	Symbol string `json:"symbol"`
	Name   string `json:"name"` // get from nav_daily
	Market string `json:"market"`
	ID     int64  `json:"cube_id"`
}

PfBasic : the basic info of a portfolio from serveral data source, get from pf_scores

func GetPfBasic 

func GetPfBasic(symbol string) (*PfBasic, error)

GetPfBasic : get portfolio basic infomation

type PfReBalanceListPage 

type PfReBalanceListPage struct {
	Count      int64           `json:"count"`
	Page       int64           `json:"page"`
	TotalCount int64           `json:"totalCount"`
	PageList   []PfRebalanceHS `json:"list"`
	MaxPage    int64           `json:"maxPage"`
}

PfReBalanceListPage : a page of rebalance records of a portfolio

func GetPfRebalanceListPage 

func GetPfRebalanceListPage(reqParams PfRebalanceParams) (*PfReBalanceListPage, error)

GetPfRebalanceListPage : get portfolio rebalance action list in history, paged

type PfRebalanceHS 

type PfRebalanceHS struct {
	Status               string             `json:"status"` //success, failed, canceled
	CashValue            float64            `json:"cash_value"`
	RebalancingHistories []PfRebalanceStock `json:"rebalancing_histories"`
	UpdatedAt            int64              `json:"updated_at"`
}

PfRebalanceHS : an item of rebalance record of a portfolio

type PfRebalanceParams 

type PfRebalanceParams struct {
	CubeSymbol string
	Count      int64 // count <= 50
	Page       int64 // from 1 to 20
}

PfRebalanceParams :

type PfRebalanceStock 

type PfRebalanceStock struct {
	StockName    string  `json:"stock_name"`
	StockSymbol  string  `json:"stock_symbol"`
	Weight       float32 `json:"weight"`
	TargetWeight float32 `json:"target_weight"`
	Price        float32 `json:"price"`
	UpdatedAt    int64   `json:"updated_at"`
}

PfRebalanceStock : a stock rebalancing of a record

type PfValueHS 

type PfValueHS struct {
	TimeStamp int64 `json:"time"`
	//DateStr   string  `json:"date"`
	Value   float32 `json:"value"`
	Percent float32 `json:"percent"`
}

PfValueHS : a net value item in a list of a portfolio

type PfValueListHS 

type PfValueListHS struct {
	Symbol string      `json:"symbol"`
	Name   string      `json:"name"`
	ListHS []PfValueHS `json:"list"`
}

PfValueListHS : the net values list of a portfolio

func GetPfValueListHS 

func GetPfValueListHS(reqParams PfValuesParams) (*PfValueListHS, error)

GetPfValueListHS : get portfolio value list in history

type PfValuesParams 

type PfValuesParams struct {
	CubeSymbol string
	Since      time.Time
	Until      time.Time
}

PfValuesParams :

type StockBasic 

type StockBasic struct {
	Symbol        string `json:"symbol"`
	Exchange      string `json:"exchange"`
	Code          string `json:"code"`
	Name          string `json:"name"`
	CurrencyUnit  string `json:"currency_unit"`
	TotalShares   uint64 `json:"totalShares"`
	UpdateBasicAt uint64 `json:"updateAt"`
}

StockBasic : basic info from real time info, all for indexes like SH000300

type StockKListParams 

type StockKListParams struct {
	Symbol     string
	Period     string
	FuquanType string
	Begin      time.Time
	End        time.Time
}

StockKListParams :

type StockMinutesParams 

type StockMinutesParams struct {
	Symbol string
	Period string
	OneMin int
}

StockMinutesParams :

type StockPriceHS 

type StockPriceHS struct {
	Volume     uint64  `json:"volume"`
	Turnrate   float32 `json:"turnrate"`
	Open       float32 `json:"open"`
	Close      float32 `json:"close"`
	High       float32 `json:"high"`
	Low        float32 `json:"low"`
	Change     float32 `json:"chg"`
	Percentage float32 `json:"percent"`
	MA5        float32 `json:"ma5"`
	MA10       float32 `json:"ma10"`
	MA20       float32 `json:"ma20"`
	MA30       float32 `json:"ma30"`
	MACD       float32 `json:"macd"`
	DEA        float32 `json:"dea"`
	DIF        float32 `json:"dif"`
	Time       string  `json:"time"`
}

StockPriceHS : Stock Price(K) in HiStory

type StockPriceListHS 

type StockPriceListHS struct {
	Success     string         `json:"success"`
	PriceListHS []StockPriceHS `json:"chartlist"`
}

StockPriceListHS : contains K price list

func GetStockPriceListHS 

func GetStockPriceListHS(reqParams StockKListParams) (*StockPriceListHS, error)

GetStockPriceListHS : get stock price(K) list in history

type StockPriceMin 

type StockPriceMin struct {
	Volume   uint64  `json:"volume"`
	AvgPrice float32 `json:"avg_price"`
	Current  float32 `json:"current"`
	Time     string  `json:"time"`
}

StockPriceMin : Stock Price(minutes in recent day) in HiStory

type StockPriceMins 

type StockPriceMins struct {
	Success       string          `json:"success"`
	PriceListMins []StockPriceMin `json:"chartlist"`
}

StockPriceMins : contains the minute price list

func GetStockPriceMinutes 

func GetStockPriceMinutes(reqParams StockMinutesParams) (*StockPriceMins, error)

GetStockPriceMinutes : get stock price minutes list in a day

type StockPriceRT 

type StockPriceRT struct {
	Current       float32 `json:"current"`
	Percentage    float32 `json:"percentage"`
	Change        float32 `json:"change"`
	Open          float32 `json:"open"`
	Close         float32 `json:"close"`
	LastClose     float32 `json:"last_close"`
	High          float32 `json:"high"`
	Low           float32 `json:"low"`
	MarketCapital float32 `json:"marketCapital"`
	RiseStop      float32 `json:"rise_stop"`
	FallStop      float32 `json:"fall_stop"`
	Volume        float32 `json:"volume"`
	PELYR         float32 `json:"pe_lyr"`
	PETTM         float32 `json:"pe_ttm"`
	EPS           float32 `json:"eps"`
	PSR           float32 `json:"psr"`
	PB            float32 `json:"pb"`
	Divident      float32 `json:"dividend"`
	UpdateAt      uint64  `json:"updateAt"`
}

StockPriceRT : real time price for now

type StockRT 

type StockRT struct {
	StockBasic
	StockPriceRT
}

StockRT : Stock RealTime info

func GetStockRT 

func GetStockRT(stockStr string) (*StockRT, error)

GetStockRT : get stock current status

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.