README
¶
erkie/rss
This is a library based on https://github.com/slymarbo/rss. It is a fork with backwards incompatible changes and fixes to the original repo.
Changes
- Changes to the
Feedstruct, removing unread flags and more - Changes to feed parsing, for compatibility reasons
- ... And more. This README was written a long time after this fork was created. So feel free to check the history
Fixes
- Charset decoding. Be a lot more lenient than standard Go towards faulty or broken encodings
Repo
Documentation
¶
Overview ¶
Package rss is a small library for simplifying the parsing of RSS and Atom feeds.
The package could do with more testing, but it conforms to the RSS 1.0, 2.0, and Atom 1.0 specifications, to the best of my ability. I've tested it with about 15 different feeds, and it seems to work fine with them.
If anyone has any problems with feeds being parsed incorrectly, please let me know so that I can debug and improve the package.
Example usage:
```go package main
import "github.com/erkie/rss"
func main() {
feed, err := rss.Fetch("http://example.com/rss")
if err != nil {
// handle error.
}
// ... Some time later ...
err = feed.Update()
if err != nil {
// handle error.
}
}
```
The output structure is pretty much as you'd expect:
```go
type Feed struct {
Nickname string // This is not set by the package, but could be helpful.
Title string
Description string
Link string // Link to the creator's website.
UpdateURL string // URL of the feed itself.
Image *Image // Feed icon.
Items []*Item
Refresh time.Time // Earliest time this feed should next be checked.
}
type Item struct {
Title string
Summary string
Content string
Link string
Date time.Time
ID string
Read bool
}
type Image struct {
Title string
Url string
Height uint32
Width uint32
}
```
The library does its best to follow the appropriate specifications and not to set the Refresh time too soon. It currently follows all update time management methods in the RSS 1.0, 2.0, and Atom 1.0 specifications. If one is not provided, it defaults to 10 minute intervals. If you are having issues with feed providors dropping connections, please let me know and I can increase this default, or you can increase the Refresh time manually. The Feed.Update method uses this Refresh time, so if Update seems to be returning very quickly with no new items, it's likely not making a request due to the provider's Refresh interval.
Index ¶
- Constants
- Variables
- func CharsetReader(theCharset string, input io.Reader) (io.Reader, error)
- func DiscardInvalidUTF8IfUTF8(input []byte, responseHeaders http.Header) []byte
- func ParseTime(s string) time.Time
- type Author
- type Categories
- type Category
- type Enclosure
- type Feed
- type Item
- type Link
- type MediaCommunity
- type MediaContent
- type MediaDescription
- type MediaGroup
- type MediaRating
- type MediaStarRating
- type MediaStatistics
- type MediaThumbnail
- type Metadata
- type ParseOptions
- type ParserFunc
- type Title
Constants ¶
const DATE = "15:04:05 MST 02/01/2006"
DATE is a date
Variables ¶
var TimeLayouts = []string{ "Mon, _2 Jan 2006 15:04:05", "Mon, _2 Jan 2006 15:04:05 MST", "Mon, _2 Jan 2006 15:04:05 Z", "Mon, _2 Jan 06 15:04:05 MST", "Mon, _2 Jan 2006 15:04:05 -0700", "Mon, _2 Jan 06 15:04:05 -0700", "_2 Jan 2006 15:04:05 MST", "_2 Jan 06 15:04:05 MST", "_2 Jan 2006 15:04:05 -0700", "_2 Jan 06 15:04:05 -0700", "2006-01-02 15:04:05", "Jan _2, 2006 15:04 PM MST", "Jan _2, 06 15:04 PM MST", time.ANSIC, time.UnixDate, time.RubyDate, time.RFC822, time.RFC822Z, time.RFC850, time.RFC1123, time.RFC1123Z, time.RFC3339, time.RFC3339Nano, "2006-01-02T15:04:05", "02-Jan-2006 15:04:05", }
TimeLayouts is contains a list of time.Parse() layouts that are used in attempts to convert item.Date and item.PubDate string to time.Time values. The layouts are attempted in ascending order until either time.Parse() does not return an error or all layouts are attempted.
Functions ¶
func CharsetReader ¶
CharsetReader is a lenient charset reader good for web inputs
func DiscardInvalidUTF8IfUTF8 ¶
DiscardInvalidUTF8IfUTF8 checks if input specifies itself as UTF8, and then runs a check to discard XML-invalid characters (because go xml parser throws up if present)
Types ¶
type Author ¶
type Categories ¶
type Categories struct {
Category `xml:"category"`
}
type Enclosure ¶
type Enclosure struct {
URL string `json:"url"`
Type string `json:"type"`
Length string `json:"length"`
}
Enclosure holds enclosure data
type Feed ¶
type Feed struct {
Type string
Title Title
Description string
Link string // Link to the creator's website.
UpdateURL string // URL of the feed itself.
Items []*Item
Links []*Link
Categories []string
}
Feed is the top-level structure.
type Item ¶
type Item struct {
Title Title `json:"title"`
Summary string `json:"summary"`
Content string `json:"content"`
Link string `json:"link"`
Date time.Time `json:"date"`
ID string `json:"id"`
Enclosures []*Enclosure `json:"enclosures"`
Meta map[string]string `json:"meta"`
Metadata
}
Item represents a single story.
func (*Item) EnsureLinks ¶
func (i *Item) EnsureLinks()
func (*Item) SetMetadata ¶
type MediaCommunity ¶
type MediaCommunity struct {
StarRating *MediaStarRating `xml:"starRating"`
Statistics *MediaStatistics `xml:"statistics"`
}
type MediaContent ¶
type MediaContent struct {
URL string `xml:"url,attr"`
Type string `xml:"type,attr"`
Width string `xml:"width,attr"`
Height string `xml:"height,attr"`
Medium string `xml:"medium,attr"`
Rating *MediaRating `xml:"rating"`
Description *MediaDescription `xml:"description"`
}
func (MediaContent) IsImage ¶
func (m MediaContent) IsImage() bool
type MediaDescription ¶
type MediaGroup ¶
type MediaGroup struct {
Content []*MediaContent `xml:"content"`
Thumbnail *MediaContent `xml:"thumbnail"`
Description *MediaDescription `xml:"description"`
Community *MediaCommunity `xml:"community"`
}
type MediaRating ¶
type MediaRating struct {
Scheme string `xml:"scheme,attr"`
}
type MediaStarRating ¶
type MediaStatistics ¶
type MediaStatistics struct {
Views string `xml:"views,attr"`
}
type MediaThumbnail ¶
type Metadata ¶
type Metadata struct {
Authors []Author `xml:"author"`
Categories []Category `xml:"category"`
MediaGroups []MediaGroup `xml:"group"`
MediaContent []MediaContent `xml:"content"`
PubDate string `xml:"pubDate"`
Date string `xml:"date"`
Published string `xml:"published"`
Updated string `xml:"updated"`
}
func (*Metadata) CategoriesAsString ¶
func (*Metadata) MediaContents ¶
func (*Metadata) PublishedDate ¶
type ParseOptions ¶
type ParserFunc ¶
type ParserFunc func(data []byte, options ParseOptions) (*Feed, error)
ParserFunc is the interface for a parser
Source Files