Extension Modules and Basic Applications

Moderator: Aaron Watters
Date: Wed May 24
URL: http://www.python.org/misc/workshop5-95/extModulesNotes.html

Work in Progress Reports

Python and the Web (Guido van Rossum)

Recently took the html parser from Tim Berners Lee and put it in 1.2
You can write an extensible WWW browser in Python
Can also easily write a robot that runs the web (Infoseek has done this in Python)
Safe Python
- Allow you to take a python program from the web and run it in a restricted environment
- One thing needed is an uncatchable exception so you can send an alarm to a running program to terminate it
Also working on a HTTP server class
- work involves putting CGI functionality directly in the server for performance
- none of this is completely working
- planning on release for 1.3
- Might also be nice to redo imagemaps in a better way (e.g. better way to determine regions)
Already a client class and a CGI class
Also working on remote import
- it would be really nice to have URL's on sys.path
- efficiency could be a problem if the URL's were towards the beginning of the path, especially if the net is slow or the server is down
- ...so caching is needed, currently underway
- negative caching also (e.g. the server did *not* have the module)
- There are curently there levels of confidence
  - level 0, believe the cache
  - level 1, check the dates if they are diffenent, get the file
  - level 2, just get the file
- Question to the audience, How do you trust a remote import?
  - ... no good answers, standard network authentication
  - this doesn't prevent a trojan horse
- POST currently calls a CGI script
CNRI work on URN's handle server project
- URN's are sort of location independant URL's
- URN's just name the document uniquely and a server gives you the actual location of the document
- CNRI currently has working handle servers
- Guido's been working on a handle server interface in Python

SQL in Python, Python in SQL (Aaron Watters)

All the spare moments I have been implementing a prototype database manager based on Python (e.g. SQL implemented in python)
SQL engines are very expensive
want to get this out for free
a filled-out implementation count be, if not a killer-app, at least used in the universitys
Right now in the form of a proof-of-concept demo
currently parsing is takes as long as the actual query
currently has the SELECT statement working
if it was ALL in python it would be slower
Futures
- interfaces to existing databases
- more SQL
- timestamping, more...
Also working on a different project that would present a bunch of python classes as relation database information. (e.g. Object oriented view of relational database)

Andrew Kuchlings's Python Cryptographer's toolkit (Ken Manheimer)

Pretty substantial collection of cryptography functions written in C with Python Class wrappers - so probably pretty fast
Got into this work as way of writing a python one-time password generator based on a shared secret DES key
kit contains a bunch of encryption algorithm's, hashing functions, and even some encoumbered code that forbids commercial use
GNU public licence currently in use
talking about writing wrappers for kerberos and Netscape's SSL

Khoros module - an image processing package (Raymond Suorsa - NASA)

Khoros Overview
- Huge image processing system
- can find edges, etc.
- current version is 2.0
- free on the net
- good a working on large datasets (e.g. the whole dataset does not have to be in memory)
- Organization
  - presentation layer
  - data pipeline
  - pysical layer
- Khiros interface
  - Kroutines - unix shell interface
    - Pro - all of Khoros in one shot
    - Pro - no recompilation needed during testing
    - Con - costly execution
    - Con - aggressive dynamic binding is probably a bad thing
  - Klibrary - c library interface - most efficient
    - Pro - fast execution with explicit binding
    - Con - 25 Megs of source
    - Con - Khorosmodule.c becomes HUGE

Python/Khoros Interface

simple example...

	import khoros
	lenna = khoros.get("lenna.xv")
	edgeLenna = khoros.kr("ingradient", args...)

Can perform some degree of operations on kr objects

	lennaConCat = lenna + edgeLenna 	# add images together

What's next
- Get Khoral research interested
- incorporate multidimentional slicing with the ':' notation

Fortran Matrix extension (Jim Fulton - USGS)

Able to call fortran
interface is simplified
error checking compatible with python exceptions
fortran routings handle input arrays polymorphically
simple built-in matrix type included
Documentation strings
fidl - provides a mechanism for calling fortran routines from Python
- each fortran module is described in a fidl spec
- each fortran routine has a fidl spec
- the fidl compiler generates the python stubs from the specs
Matrices
- similar to array objects
- homogeneous data
- n-dimensional arrays behave like sequences of n-1 dimentinal arrays
- foo[i][j][k] = spam - works and works efficiently
import interp b,c - dcsakm([0, .1, .2, .5], [100, 200, 10, 50]) for i in range(50): print dcsval(i/100.0,b,c)

Note the Doc string in the function...

	Fit an Akima spline function

	Usage: csbreak, cscoef = dcsakm(xdata, fdata)

MMACE's application execution control panel (Michael McLay)

Design tools for the MicroWave and Millimeter industry
Using python for a nice front-end
Demo...

Career Web (Eric Bouck)

Overview Presentation

InfoSeek (Andy Benskey & Jim Roskind- InfoSeek)

Home Page
Used python after trying to implement an http server in perl
modules written to handle accounting, etc
searches are actually done in C
python threads were not used - heavy weight forks used instead
- problem was with reaping child processes due to some signal wierdness
- used a dictionary stuffed with function calls... nice alternative to if-then-if - much faster too
- would be nice to have sys.environ more tightly coupled with the actual environment
- Guido - 'I hear you'
- developed a persistent object store
  - we want an object to survive beyond the process boundry
  - pobject.py
  - to deal with type evolution one overrides methods derived from a base class that was a persistent object
  - Examples
```
	class M(pobject):
	  def ...
	m = M()
	m.SetAlias('jar')
	m2.FindAlias('jar')
	m2.Find('jar')
	m.GetID()
      
```
  - store an object in human-readable form... so above would be file: M..jar wich has a very python-dictionary like contents
  - much like pickle
  - supports the notion of polymorhphism
  - strong relationship between naming and persistence
  - implementation handles recursion
```
	   x = []
	   x.append(x)
	   # now archive this!
	
```
- developed a profiler with statistics generation capability so one could determine where python is spending its time
  - why is this function being called 1500 times?
  - profile.py
  - ```
   run(" program ") 
```
- pstats.py
- pstats.sort_stats().print_stats() # sorting and printing statistics
- contributing both to python distribution

The PLplot module (Michael McLay)

Part of the MMACE demo...

Identifying important extensions, current and future (Michael McLay)

contributing modules to the PSA

Executable HTML documents (Diane Hackborn - Oregon State University)

Information page
Interface does not depend on python
Another language could be dropped in cleanly
Currently uses sockets to go back to the server (needs to be replaced with URL's)
Python interpreter is linked into the browser for performance