plotting multiple sampled data points from a log

I have a log file which I need to plot in python with different data points as a multi line plot with a line for each unique point , the problem is that in some samples some points would be missing and new points would be added in another, as shown is an example with each line denoting a sample of n points where n is variable:

2015-06-20 16:42:48,135 current stats=[ ('keypassed', 13), ('toy', 2), ('ball', 2),('mouse', 1) ...] 2015-06-21 16:42:48,135 current stats=[ ('keypassed', 20, ('toy', 5), ('ball', 7), ('cod', 1), ('fish', 1) ... ]

in the above 1 st sample 'mouse ' is present but absent in the second line with new data points in each sample added like 'cod','fish'

so how can this be done in python in the quickest and cleanest way? are there any existing python utilities which can help to plot this timed log file? Also being a log file the samples are thousands in numbers so the visualization should be able to properly display it.

Interested to apply multivariate hexagonal binning to this and different color hexagoan for each unique column "ball,mouse ... etc". scikit offers hexagoanal binning but cant figure out how to render different colors for each hexagon based on the unique data point

extract comments and hyper links from cells of *.xls file using xlrd library in python

I want to extract comments and hyperlinks from cells of *.xls file using xlrd library in python in windows. Please let me know the functions and sample code.

Thanks in advance.

call python from matlab2015a

As matlab2015a has been able to support python. But it seems it can only call standard library in matlab. If I want to import other library such as numpy, scipy or sklearn, what should I do? And can I execute the python script directly. Unfortunately, the offical document of matlab has not given enough explanations. If anyone can explain, I will be really appreciated!

Python programming for card game War

This program is supposed to play the card game War. Okay so I need to be able to prompt the user to keep playing and have them respond by hitting the enter key. I'm not sure how to make this happen. Help?

import cards

# Create a deck of cards
the_deck = cards.Deck()

# Shuffle the deck, then display it in 13 columns
the_deck.shuffle()
print( "===== shuffled deck =====" )
the_deck.display()


def main():
    '''This function deals out half the deck to each
    player. It sorts the rank so as to solve the problem
    with Aces causing an infinite game'''
    player1_list=[]
    player2_list=[]
    for i in range( 26 ):
        p1_temp= the_deck.deal()
        player1_list.append( p1_temp )
        p2_temp= the_deck.deal()
        if (p2_temp.rank()==1):
            player1_list.append(p2_temp)
            player2_list.append(player1_list.pop(0))
        else:
            player2_list.append(p2_temp)


    print()
    # Card dealt to Player #1
    player1_card = player1_list.pop( 0 )
    print( "===== player #1 =====" )
    print( "Card dealt to player #1:", player1_card )
    print( player1_list )
    print()
    #Card dealt to Player #2
    player2_card=player2_list.pop(0)
    print( "===== player #2 =====" )
    print("Card dealt to player #2:",player2_card)
    print( player2_list )

    # Compare the two cards using overloaded operators
    print()
    if player1_card == player2_card:
        print( "Tie:", player1_card, "and", player2_card, "of equal rank" )
    elif player1_card > player2_card:
        print("Player #1 wins:",player1_card,"of higher rank than",player2_card)
    else:
        print("Player #2 wins:",player2_card,"of higher rank than",player1_card)
        print()
main()
def keep_playing():
    '''Determines whether the player wants to continue. If so they press the
    Enter key and the function calls main to start all over.'''
    still_playing=input('Press "Enter" to continue playing')
    Enter=1
    while still_playing==Enter:
        main()
keep_playing()

Error import spatial data in GeoDjango - KeyError for mpoly field

I was following the tutorial on http://ift.tt/1NnlWfX for setting up GeoDjango on my machine. But it seems like there is some issue there. While importing data using LayerMapping using load.run() from python shell, I get the following error:

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/home/ubuntu/src/django/world/load.py", line 23, in run
    lm = LayerMapping(WorldBorder, world_shp, world_mapping, transform=False, encoding='iso-8859-1')
  File "/home/ubuntu/Envs/vir-env/local/lib/python2.7/site-packages/django/contrib/gis/utils/layermapping.py", line 105, in __init__
    self.check_layer()
  File "/home/ubuntu/Envs/vir-env/local/lib/python2.7/site-packages/django/contrib/gis/utils/layermapping.py", line 178, in check_layer
    ogr_field_types = self.layer.field_types
  File "/home/ubuntu/Envs/vir-env/local/lib/python2.7/site-packages/django/contrib/gis/gdal/layer.py", line 153, in field_types
    for i in range(self.num_fields)]
KeyError: 12

Then I found out that, there is no 'MULTIPOLYGON' field in the .shp file:

>>> from django.contrib.gis.gdal import DataSource
>>> ds = DataSource('world/data/TM_WORLD_BORDERS-0.3.shp')
>>> layer = ds[0]
>>> layer.fields
[u'FIPS', u'ISO2', u'ISO3', u'UN', u'NAME', u'AREA', u'POP2005', u'REGION', u'SUBREGION', u'LON', u'LAT']

So, definitely in the world_mapping file, importing will fail for the 'mpoly': 'MULTIPOLYGON' mapping. Has anyone else faced this issue? I hope so, as I've followed the tutorial step-by-step. But it doesn't say anything about such issue.

Here's my load.py file:

  1 import os
  2 from django.contrib.gis.utils import LayerMapping
  3 from models import WorldBorder
  4
  5 world_mapping = {
  6     'fips' : 'FIPS',
  7     'iso2' : 'ISO2',
  8     'iso3' : 'ISO3',
  9     'un' : 'UN',
 10     'name' : 'NAME',
 11     'area' : 'AREA',
 12     'pop2005' : 'POP2005',
 13     'region' : 'REGION',
 14     'subregion' : 'SUBREGION',
 15     'lon' : 'LON',
 16     'lat' : 'LAT',
 17     'mpoly' : 'MULTIPOLYGON',
 18 }
 19
 20 world_shp = os.path.abspath(os.path.join(os.path.dirname(__file__), 'data/TM_WORLD_BORDERS-0.3.shp'))
 21
 22 def run(verbose=True):
 23     lm = LayerMapping(WorldBorder, world_shp, world_mapping, transform=False, encoding='iso-8859-1')
 24
 25     lm.save(strict=True, verbose=verbose)

Xpath extract dates between certain characters AND use as dates

UPDATE: Regarding my 2nd question (how to convert string to date format in MySQL), I found a way and want to share it:

1) Save the "string date" data as VARCHAR (Don't use TEXT)

2) When showing MySQL data in PHP or other ways, use the function of str_to_date(string-date-column, date-format), such as the following example:

$sql = "SELECT * FROM yourtablename ORDER BY str_to_date(string-date-column, '%d %M %Y')";

---

I am using scrapy to collect data, write to database. From a website, the post date of each item is listed as following:

<p>   #This is the last <p> within each <div>
<br>
[15 May 2015, #9789]
<br>
</p>

So the date is always behind a "[" and before a ",". I used the following xpath code to extract:

sel.xpath("p[last()]/text()[contains(., '[')]").extract()

But I will get the whole line:

[15 May 2015, #9789]

So, how to get only the part of "15 May 2015"? If this can be done, how to convert the scraped string (15 May 2015) as real DATE data, so it can be used for sorting? Thanks a bunch!

How to save a dataframe as a csv file with '/' in the file name

I want to save a dataframe to a .csv file with the name '123/123', but it will split it in to two strings if I just type like df.to_csv('123/123.csv').

Anyone knows how to keep the slash in the name of the file?