4 years agoMerge branch 'master' into topic/multi topic/multi
Michael D Labriola [Sun, 7 Jul 2013 23:56:59 +0000]
Merge branch 'master' into topic/multi

* master:
  added musings regarding multiprocessing, v2
  updated docs regarding extensible features

4 years agoadded musings regarding multiprocessing, v2
Michael D Labriola [Sun, 7 Jul 2013 23:56:17 +0000]
added musings regarding multiprocessing, v2

4 years agoupdated docs regarding extensible features
Michael D Labriola [Sun, 7 Jul 2013 23:55:52 +0000]
updated docs regarding extensible features

4 years agofeatures/core: multiprocessing for install_func
Michael D Labriola [Sat, 6 Jul 2013 21:09:17 +0000]
features/core: multiprocessing for install_func

This is definately not done yet, but i think i want it to work something
like this.

4 years agoadded musings regarding multiprocessing
Michael D Labriola [Sat, 6 Jul 2013 14:40:53 +0000]
added musings regarding multiprocessing

4 years agofeatures/core: fleshed out install_func, v3
Michael D Labriola [Mon, 22 Apr 2013 01:22:30 +0000]
features/core: fleshed out install_func, v3

We're now installing manifest (which i'm going to go back and consistently
refer to as db) entry in /var/lib/srp/pkgname/SHA.  So far it's NOTES and
FILES.

Still a bunch of FIXMEs and half-implemented stuff...  upgrade logic, for
example is completely missing.  I should probably also go back and recreate
a fobj for the NOTES file at the very end of db entry creation so we can
update the NOTES file during installation.  And perhaps we should create
the db as yet another compressed tarball as apposed to a dir of files?

4 years agofeatures/core: fleshed out install_func, v2
Michael D Labriola [Sun, 21 Apr 2013 03:15:37 +0000]
features/core: fleshed out install_func, v2

Started playing with pickle, which looks good.  I think we'll install a
pickled list of TarInfo in /var/lib/srp/pkgname/sha.  Other features can
install other files in there as well (e.g., checksum).

4 years agocli,features/core: reworking SHA handling in package files, v2
Michael D Labriola [Sun, 21 Apr 2013 02:29:51 +0000]
cli,features/core: reworking SHA handling in package files, v2

Actually got around to testing this and ironed out the problems I knew I
would find (typos, brain farts, etc).

4 years agocli,features/core: reworking SHA handling in package files
Michael D Labriola [Thu, 18 Apr 2013 03:10:10 +0000]
cli,features/core: reworking SHA handling in package files

This is incomplete and untested... but it's late and I'm tired.

4 years agofeatures/core: fleshed out install_func
Michael D Labriola [Thu, 18 Apr 2013 03:08:55 +0000]
features/core: fleshed out install_func

So far, we're extracting the files.  We still need to record the package
manifest.

4 years agonotes: added blob_compression to check method
Michael D Labriola [Thu, 18 Apr 2013 03:07:16 +0000]
notes: added blob_compression to check method

4 years agofeatures/deps: fleshed out install_func
Michael D Labriola [Thu, 18 Apr 2013 03:06:39 +0000]
features/deps: fleshed out install_func

4 years agocli: fixed optional argument parsing
Michael D Labriola [Thu, 18 Apr 2013 03:04:48 +0000]
cli: fixed optional argument parsing

4 years agofeatures: added optional feature dependencies
Michael D Labriola [Sun, 24 Mar 2013 20:36:58 +0000]
features: added optional feature dependencies

This commit allows stage_structs to be created with features listed as deps
preceded with a question mark to indicate deps rules if a feature is
enabled, w/out recursively enabling it.  For example, if checksum is
enabled, strip_debug has to come first, but if it's not, strip_debug
doesn't care.

4 years agofeatures: added strip_debug feature module
Michael D Labriola [Sun, 24 Mar 2013 20:30:25 +0000]
features: added strip_debug feature module

4 years agonotes: added update method
Michael D Labriola [Sun, 24 Mar 2013 20:29:46 +0000]
notes: added update method

This method is used to update internal notes fields after instantiation
(e.g., cli to update features as requested via command line arguments).

4 years agofeatures: added argument to install funcs
Michael D Labriola [Sun, 24 Mar 2013 20:24:31 +0000]
features: added argument to install funcs

4 years agocli: started plumbing the install code, v2
Michael D Labriola [Sun, 24 Mar 2013 20:24:00 +0000]
cli: started plumbing the install code, v2

4 years agonotes: added check method, v2
Michael D Labriola [Mon, 4 Mar 2013 03:45:46 +0000]
notes: added check method, v2

Oops, missed the usage in do_create...

4 years agocli: started plumbing the install code
Michael D Labriola [Mon, 4 Mar 2013 03:42:02 +0000]
cli: started plumbing the install code

4 years agotoc: removed
Michael D Labriola [Mon, 4 Mar 2013 03:41:45 +0000]
toc: removed

4 years agonotes: added check method
Michael D Labriola [Mon, 4 Mar 2013 03:41:33 +0000]
notes: added check method

This new method raises an exception if srp version or feature list is
inadequate for the NOTES file being parsed.

This commit also fixes notes usage in cli.

4 years agoconfig: added config submodule
Michael D Labriola [Mon, 4 Mar 2013 03:39:02 +0000]
config: added config submodule

4 years agofeatures/deps: initial commit
Michael D Labriola [Mon, 4 Mar 2013 02:47:08 +0000]
features/deps: initial commit

This new feature has a working build func that adds deps to the NOTES file
during build.  The install_func still needs to be fleshed out

4 years agocli,features: toned down debug
Michael D Labriola [Mon, 4 Mar 2013 02:45:57 +0000]
cli,features: toned down debug

4 years agoexample: removed env from build script
Michael D Labriola [Mon, 4 Mar 2013 02:44:36 +0000]
example: removed env from build script

4 years agoexample: link against libreadline and libncurses
Michael D Labriola [Sun, 3 Mar 2013 23:51:40 +0000]
example: link against libreadline and libncurses

This commit makes our silly example executable link against a few libraries
so we can test the deps feature.

4 years agocli: working on do_build, v2
Michael D Labriola [Sun, 3 Mar 2013 12:39:11 +0000]
cli: working on do_build, v2

We're now actually writing things correctly (uncompressed tar containing
NOTES and compressed tar) and autodetecting machine architecture for the
brp name.

4 years agonotes: added write method, v2
Michael D Labriola [Sun, 3 Mar 2013 12:36:50 +0000]
notes: added write method, v2

4 years agocli: working on do_build
Michael D Labriola [Fri, 1 Mar 2013 04:06:24 +0000]
cli: working on do_build

This commit isn't finished, but does make it so we can at least write out
a misnamed tarball to verify that the perms build_func is working.

4 years agofeatures/perms: started working on build method, v3
Michael D Labriola [Fri, 1 Mar 2013 04:03:34 +0000]
features/perms: started working on build method, v3

Fixed a few minor issues.  This actually works now.

4 years agoexample: added more complex perms rules
Michael D Labriola [Fri, 1 Mar 2013 04:01:44 +0000]
example: added more complex perms rules

This commit amps up the complexity of our perms section for testing setuid,
symlinks, etc.

4 years agoexample: added symlinks to installed files
Michael D Labriola [Fri, 1 Mar 2013 03:58:51 +0000]
example: added symlinks to installed files

This commit adds a symlink to a file and a symlink to a dir to the install
target.  This is for testing behavior of srp's installation routines that
need to do special things for regular files vs dirs vs symlinks, etc.

4 years agofeatures/perms: started working on build method, v2
Michael D Labriola [Fri, 1 Mar 2013 01:15:27 +0000]
features/perms: started working on build method, v2

This appears to be all set.  Pattern matching works and payload files have
their TarInfo objects modifed as desired.

4 years agofeatures/perms: started working on build method, v1
Michael D Labriola [Thu, 28 Feb 2013 01:29:06 +0000]
features/perms: started working on build method, v1

This isn't done yet, but it's fleshed out nicely.  Moved the old 'new'
owneroverride class into the feature module (named it perms) and set up
the build_func to iterate over payload files examining the regular
expressions from the NOTES file's perms section.  Matching seems to work,
now to actually modify the manifest's TarInfo objects.

4 years agofeatures/core: started working on build method, v5
Michael D Labriola [Thu, 28 Feb 2013 01:26:32 +0000]
features/core: started working on build method, v5

Changed manifest into a map so we can pull things out easily later (e.g.,
in the perms build_func)

4 years agofeatures/core: started working on build method, v5
Michael D Labriola [Wed, 27 Feb 2013 19:44:22 +0000]
features/core: started working on build method, v5

Updated v4 patch to use new notes.additions map and write method

4 years agonotes: added write method
Michael D Labriola [Wed, 27 Feb 2013 19:42:23 +0000]
notes: added write method

This is needed so that once we've added things to the NOTES file instance
during build, we can actually create a filobj to add to the brp archive.

4 years agofeatures/core: started working on build method, v4
Michael D Labriola [Mon, 25 Feb 2013 01:54:40 +0000]
features/core: started working on build method, v4

This is trickling into cli and notes as well now, because I need to be able
to update the NOTES file with extra build-specific data.

4 years agofeatures/core: started working on build method, v3
Michael D Labriola [Mon, 25 Feb 2013 00:49:28 +0000]
features/core: started working on build method, v3

We're now extracting sources, creating and running build script, and
recording TarInfo objects of all payload files.  Still need to add brp
lines to NOTES file.

4 years agoexample: fixed build script
Michael D Labriola [Mon, 25 Feb 2013 00:45:08 +0000]
example: fixed build script

4 years agofeatures/core: started working on build method, v2 rethink
Michael D Labriola [Sun, 10 Feb 2013 21:09:23 +0000]
features/core: started working on build method, v2

Still nowhere near done.  This commit even leaves off with 2 half-done
lines that won't parse in features/core.py

4 years agonotes: fobj must be opened in binary mode
Michael D Labriola [Sun, 10 Feb 2013 18:52:50 +0000]
notes: fobj must be opened in binary mode

4 years agonotes: should notes be opened in binary or text mode?
Michael D Labriola [Sat, 12 Jan 2013 01:35:40 +0000]
notes: should notes be opened in binary or text mode?

I was opening in text mode (default) in the create_func, but when we pull
the fobj for NOTES out of the tar, it's opened in binary mode...  I either
have to decide that one way is the only way to do it, or add some type
checking conditionals in the notes constructor

4 years agofeatures/core: started working on build method
Michael D Labriola [Sat, 12 Jan 2013 01:33:43 +0000]
features/core: started working on build method

This is nowhere near done.  I'm still figuring out what the function
prototype for build funcs should be, plumbing the cli for --build, and
laying other groundwork for testing this out...

4 years agofeatures/core: create_func writes to appropriately named file
Michael D Labriola [Sat, 12 Jan 2013 01:28:28 +0000]
features/core: create_func writes to appropriately named file

4 years agofeatures/perms: move tarinfo forging to build stage
Michael D Labriola [Sun, 6 Jan 2013 04:03:34 +0000]
features/perms: move tarinfo forging to build stage

4 years agofeatures/core: working on create method, v2
Michael D Labriola [Sun, 6 Jan 2013 03:59:39 +0000]
features/core: working on create method, v2

Fixed the corrupt archive problems from the previous commit and remembered
to add NOTES to the archive.  This seems to be working now.

4 years agofeatures/core: working on create method
Michael D Labriola [Sat, 5 Jan 2013 02:20:01 +0000]
features/core: working on create method

This doesn't quite work yet.  No exceptions, but the resulting tarball is
corrupt... sigh.

4 years agonotes: features can be enabled by adding NOTES section
Michael D Labriola [Fri, 4 Jan 2013 23:31:04 +0000]
notes: features can be enabled by adding NOTES section

This commit allows for implicit feature enabling by assuming that all
section headers correspond with a feature name (except for the default
sections).

In other words, if you add a [perms] section, it automatically adds perms
to the required feature list.

NOTE: This commit also changes the notes.options.flags map into the
      notes.options.features list because we're going to use feature
      sections in the NOTES file to pass arguments/variables into the
      various feature funcs.

4 years agofeatures: added default item to feature_struct
Michael D Labriola [Fri, 4 Jan 2013 18:50:07 +0000]
features: added default item to feature_struct

This commit makes register_feature populate our list of default features
(instead of specifying them statically).

It also moves the action_map generation to register_feature.

4 years agofeatures: reworked function_list and stage_map code, v2
Michael D Labriola [Fri, 4 Jan 2013 18:31:49 +0000]
features: reworked function_list and stage_map code, v2

4 years agomisc cleanup
Michael D Labriola [Fri, 4 Jan 2013 18:31:11 +0000]
misc cleanup

4 years agofeatures: reworked function_list and stage_map code
Michael D Labriola [Fri, 4 Jan 2013 18:22:55 +0000]
features: reworked function_list and stage_map code

4 years agonotes: handle default flags and inverse flags
Michael D Labriola [Fri, 4 Jan 2013 18:21:40 +0000]
notes: handle default flags and inverse flags

4 years agofeatures: initial commit of core, perms, and checksum
Michael D Labriola [Fri, 4 Jan 2013 18:19:00 +0000]
features: initial commit of core, perms, and checksum

4 years agofeatures/core: initial commit
Michael D Labriola [Mon, 31 Dec 2012 18:43:44 +0000]
features/core: initial commit

4 years agocli: added required feature check to do_create
Michael D Labriola [Mon, 31 Dec 2012 18:43:06 +0000]
cli: added required feature check to do_create

4 years agoupdated doc strings
Michael D Labriola [Mon, 31 Dec 2012 18:42:28 +0000]
updated doc strings

4 years agofeatures: all .py files now get autoloaded when srp.features is imported
Michael D Labriola [Mon, 31 Dec 2012 16:08:25 +0000]
features: all .py files now get autoloaded when srp.features is imported

4 years agomodules/cli: started fleshing out package creation
Michael D Labriola [Fri, 21 Dec 2012 21:56:12 +0000]
modules/cli: started fleshing out package creation

4 years agoexample: renamed NOTES file
Michael D Labriola [Fri, 21 Dec 2012 21:54:09 +0000]
example: renamed NOTES file

4 years agonotes: initial commit
Michael D Labriola [Fri, 21 Dec 2012 21:13:01 +0000]
notes: initial commit

4 years agofeatures: added feature_struct
Michael D Labriola [Fri, 21 Dec 2012 17:26:27 +0000]
features: added feature_struct

4 years agomodules: cleanup
Michael D Labriola [Fri, 21 Dec 2012 16:11:25 +0000]
modules: cleanup

4 years agofeatures: make register_feature track pre and post requirements
Michael D Labriola [Fri, 21 Dec 2012 15:32:37 +0000]
features: make register_feature track pre and post requirements

This commit is the 1st draft of an attempt to make register_feature track
not only the features that required before this one, but also the ones that
are required after it.  This is being accomplished by creating a registration
structure.

Note that this means the stage list is reduced to simply: build, install,
uninstall, and action.

4 years agoexample: cleanup
Michael D Labriola [Sat, 15 Dec 2012 18:32:32 +0000]
example: cleanup

4 years agofeatures: cleaned up a bit
Michael D Labriola [Sat, 15 Dec 2012 18:32:12 +0000]
features: cleaned up a bit

4 years agofeatures: initial commit
Michael D Labriola [Sat, 15 Dec 2012 18:31:55 +0000]
features: initial commit

4 years agoadded .gitignore
Michael D Labriola [Sat, 15 Dec 2012 18:31:30 +0000]
added .gitignore

5 years agoexample: added new autotools-based 3.0 example
Michael D Labriola [Sun, 12 Aug 2012 18:23:05 +0000]
example: added new autotools-based 3.0 example

The SRP_Files dir is copied straight out of the dev-3.0 tree for now.

5 years agosrp: commented out exception handling for now
Michael D Labriola [Wed, 27 Jun 2012 18:42:01 +0000]
srp: commented out exception handling for now

5 years agocli: added --create
Michael D Labriola [Wed, 27 Jun 2012 18:40:55 +0000]
cli: added --create

5 years agoinit: commented out some unneeded code
Michael D Labriola [Wed, 27 Jun 2012 18:40:00 +0000]
init: commented out some unneeded code

5 years agodev-3.0: commiting some old stuff
Michael D Labriola [Wed, 27 Jun 2012 18:39:08 +0000]
dev-3.0: commiting some old stuff

5 years agoreorg: starting to integrate all the dev stuff
Michael D Labriola [Wed, 7 Mar 2012 18:35:04 +0000]
reorg: starting to integrate all the dev stuff

5 years agoadded notes to myself
Michael D Labriola [Wed, 7 Mar 2012 18:00:18 +0000]
added notes to myself

5 years agoinit: tidied up more arg parsing
Michael D Labriola [Sun, 4 Mar 2012 19:36:46 +0000]
init: tidied up more arg parsing

5 years agoinit: fixed default handling and added fnmatch notes
Michael D Labriola [Sun, 4 Mar 2012 14:18:29 +0000]
init: fixed default handling and added fnmatch notes

5 years agoinit: reworked parsing to use seperate -p flag
Michael D Labriola [Sat, 3 Mar 2012 18:35:13 +0000]
init: reworked parsing to use seperate -p flag

5 years agomore argparse work
Michael D Labriola [Sat, 3 Mar 2012 14:40:29 +0000]
more argparse work

5 years agostarted playing with python3's argparser
Michael D Labriola [Wed, 29 Feb 2012 19:43:19 +0000]
started playing with python3's argparser

5 years agofeatures: converted to python3
Michael D Labriola [Wed, 29 Feb 2012 19:42:50 +0000]
features: converted to python3

6 years agoadded Feature musings
Michael D Labriola [Fri, 18 Nov 2011 19:13:18 +0000]
added Feature musings

7 years agowork: new owneroverride support prerethink
Michael D Labriola [Fri, 9 Jul 2010 00:08:28 +0000]
work: new owneroverride support

This isn't done yet, but was being tested a while ago... before life got
busy.

7 years agoowneroverride: more work
Michael D Labriola [Fri, 9 Jul 2010 00:07:34 +0000]
owneroverride: more work

I don't even remember what I was doing here... but this was staged for
commit from like 2 months ago... ;-)

7 years agoowneroverride: compile a re object for each entry
Michael D Labriola [Sat, 5 Jun 2010 00:19:28 +0000]
owneroverride: compile a re object for each entry

Instead of just storing the regex pattern in our list of rules, we now
store a precompiled re object in there.

Also changed my example v3 package's owneroverride file, as I realized that
the re module uses perl-style regex not awk-style.

7 years agoowneroverride: changed parent class to list
Michael D Labriola [Sat, 29 May 2010 20:24:44 +0000]
owneroverride: changed parent class to list

Changed to parent class of ownerovrride to list from dict.  This is
needed so we can ensure that rules are applied in the order they're
defined in the file.

Also fixed comment handling and splitting on ':' when regex contains
that char.

7 years agoadded overrides to example packages
Michael D Labriola [Sat, 29 May 2010 20:23:37 +0000]
added overrides to example packages

Ha.  My example packages didn't use SRP_OWNEROVERRIDE... guess that
hasn't been getting much testing.

7 years agorewrote owneroverride module
Michael D Labriola [Sat, 29 May 2010 19:50:34 +0000]
rewrote owneroverride module

Pretty much completely rewrote the owneroverride module, giving it
much more functionallity and flexibilty.  It now supports so much more
than changing ownership, that I might want to consider renaming the
module (and package file).

7 years agoimplemented prepostlib.v2_wrapper
Michael D Labriola [Sat, 29 May 2010 02:32:05 +0000]
implemented prepostlib.v2_wrapper

This commit removes the prepostlib.init method and adds a
prepostlib.v2_wrapper class.  Started to add similar changes to
owneroverride, but didn't finish os it's commented out.

While doing this, I also removed some dead code (notes.init, notes.v2,
etc) and fixed up my currently hardcoded config.LIBDIR variables.

7 years agofixing regressions
Michael D Labriola [Mon, 24 May 2010 02:24:32 +0000]
fixing regressions

The v2->v3 translation for source package was pretty broken... fixed a
bunch of stuff here.

- translated build scripts had CFLAGS and SRP_ROOT hardcoded in them
- notes.empty was defined twice...
- fixed notes parsing for v2 dirname runs (we did need addfile_dir!)
- added a bunch of debug code that should be removed... :-/

Now we can really instantiate v2->v3 packags from directory of files
or from previously created packages... but the builder fails to build
them.

7 years agofixed main module and abspath-ized __filename in commit
Michael D Labriola [Sun, 23 May 2010 03:58:10 +0000]
fixed main module and abspath-ized __filename in commit

7 years agoadded a skeletal main.py module
Michael D Labriola [Sun, 23 May 2010 03:26:00 +0000]
added a skeletal main.py module

7 years agomade commit method take an optional dir arg
Michael D Labriola [Sun, 23 May 2010 03:25:15 +0000]
made commit method take an optional dir arg

7 years agoreorg and cleanup
Michael D Labriola [Sun, 23 May 2010 02:00:51 +0000]
reorg and cleanup

This commit does some pretty hefty reorg.  Well, I guess not too
hefty.

- The installer, uninstaller, and builder modules have all been merged
  into a work module (work.builder, work.installer, etc).

- The classes in the package module have been renamed (srp->source,
  brp->binary) and the skeletal forms of the package.installed class
  has been created.

- Added a size method to the package.binary class.  Its return value
  is chaced in self.__size.

7 years agoremoved dead files v3.0.0-alpha1
Michael D Labriola [Fri, 21 May 2010 02:35:43 +0000]
removed dead files

Removed deps.py and files.py since we're just using a map of TarInfo
objects (with additional deps data and forged ownership) instead.

Also removed testing.py... don't really even remember what I was
testing with that.

7 years agocleaned up a bit
Michael D Labriola [Fri, 21 May 2010 02:28:46 +0000]
cleaned up a bit

Updated the README, removed some dead code, and did other misc cleanup.

7 years agofleshed out brp class
Michael D Labriola [Fri, 21 May 2010 02:03:42 +0000]
fleshed out brp class

This commit makes our builder instance successfully return a list of
brp objects.  The brp.__init__ method now can be invoked 2 different
ways to instantiate from disk or use precreated data members (from
builder).  I didn't actually implement the from-disk stuff yet, or the
rest of the brp member functions (commit, etc)

Also fixed up he srp.extract methods a bit, too.  They weren't quite
working well enough to get through the build process.

8 years agotracedmethod only applied if DEBUG on
Michael D Labriola [Thu, 3 Sep 2009 14:23:13 +0000]
tracedmethod only applied if DEBUG on

made the autodec_class metaclass only apply the tracedmethod decorator
if DEBUG is turned on