Changeset 262


Ignore:
Timestamp:
06/17/11 13:51:31 (13 years ago)
Author:
tim
Message:

changed regfi_conv_charset to handle memory allocation
tweaked test cases
corrected some documentation

Files:
5 edited

Legend:

Unmodified
Added
Removed

  • test/pyregfi-smoketest.py

    r255 r262  
    6565        path = getCurrentPath(k)
    6666        try:
     67            print(repr(path))
    6768            hive_iter = hive.subtree(path)
    6869            if hive_iter.current_key() != k:
    … …  
    174175                ssk = hive_iter.find_subkey(sk.name)
    175176                if ssk != None:
    176                     sk_stat += len(ssk.name)
     177                    if ssk.name != None:
     178                        sk_stat += len(ssk.name)
    177179                else:
    178180                    print("WARNING: ssk was None")
    … …  
    183185                vv = hive_iter.find_value(v.name)
    184186                if vv != None:
    185                     v_stat += len(vv.name)
     187                    if vv.name != None:
     188                        v_stat += len(vv.name)
    186189                else:
    187190                    print("WARNING: vv was None")
    … …  
    230233
    231234def loopSecurity(hive, fh):
    232     start = hive.root.fetch_security()
    233     print(start.descriptor.group)
    234     cur = start.next_security()
    235 
    236     while cur != start:
    237         print(start.descriptor.group)
    238         cur = cur.next_security()
     235    cur = hive.root.fetch_security()
     236    while True:
     237        stat += len(cur.descriptor.owner)
     238        stat += len(cur.descriptor.group)
     239        if cur.descriptor.sacl:
     240            stat += len(cur.descriptor.sacl)
     241        if cur.descriptor.dacl:
     242            stat += len(cur.descriptor.dacl)
     243       
     244        nxt = cur.next_security()
     245        if cur == nxt:
     246            break
    239247
    240248   
    … …  
    278286
    279287def usage():
    280     sys.stderr.write("USAGE: pyregfi-smoketest.py test1[,test2[,...]] hive1 [hive2 ...]\n")
     288    sys.stderr.write("USAGE: pyregfi-smoketest.py { test1[,test2[,...]] | * } hive1 [hive2 ...]\n")
    281289    sys.stderr.write("\tAvailable tests:\n")
    282290    for t in tests.keys():
    … …  
    288296    sys.exit(1)
    289297
    290 selected_tests = sys.argv[1].split(',')
    291 for st in selected_tests:
    292     if st not in tests:
    293         usage()
    294         sys.stderr.write("ERROR: %s not a valid test type\n\n" % st)
    295         sys.exit(1)
     298if sys.argv[1] == '*':
     299    selected_tests = tests.keys()
     300else:
     301    selected_tests = sys.argv[1].split(',')
     302    for st in selected_tests:
     303        if st not in tests:
     304            usage()
     305            sys.stderr.write("ERROR: %s not a valid test type\n\n" % st)
     306            sys.exit(1)
    296307
    297308files = []

  • trunk/doc/devel/TODO

    r250 r262  
    1515   descriptor information.  Maybe by MTIME as well.
    1616
     17 - reglookup-timeline needs to be replaced with something cross-platform. 
     18   Perhaps a python script that provides MTIME range filtering capabilities.
     19
     20 - Need to integrate much of reglookup-recover's algorithms into regfi
     21   and then expose them from the bottom-up to provide building blocks
     22   through regfi and pyregfi.  This should be addressed along with code
     23   to support handling of partial/fragmented registry hives.
     24
    1725 - Testing, testing, and more testing.  reglookup needs to be more
    1826   heavily tested on all recent Windows platforms.  A regression test
    … …  
    2634   to be decent, UTF-8 output would be nice.
    2735
    28  - Develop and solidify regfi API.  Regfi should be better documented and
    29    eventually needs a set of higher-language wrappers, starting with Python
    30    and possibly moving on to Perl as well.
     36 - Continue to improve regfi/pyregfi APIs as needed.  winsec library needs more
     37   flexibility and documentation.
     38
     39 - Consider adding regfi wrappers for other high-level languages (perl? ruby?).
    3140
    3241 - Documentation.  The security descriptor output format needs to be
    3342   documented.  Also, function contracts should be added to the
    34    lower-level functions of regfi.c.
     43   lower-level functions of regfi.c. Continue adding
    3544
    3645 - Consider switching from libiconv to Joachim Metz's libuna for
    … …  
    3948 - Grep through the source for 'XXX', and you'll find more.
    4049
     50 - Consider integrating packaging rules for debian/other platforms into trunk.
     51
     52 - Investigate why file descriptors can't be directly used in Windows
    4153
    4254
    … …  
    4456===========
    4557
    46 Add fields/methods for accessing security descriptors in pyregfi
    47 
    48 convert MTIME structure to uint64_t if possible
    49 
    50 investigate why file descriptors can't be directly used in Windows
    51 
    52 Fill in and update remaining regfi/pyregfi API documentation
    53 
    54 Possible debian package build rules
    55 
    56 Possibly replace reglookup-timeline with something cross-platform
    57 
    5858Testing
    5959  Full diffs
    6060  regfi and pyregfi threading
    6161  valgrind in multiple scenarios for reglookup, reglookup-recover
    62 
     62  double check man pages

  • trunk/include/regfi.h

    r258 r262  
    11/*
    2  * Copyright (C) 2005-2010 Timothy D. Morgan
     2 * Copyright (C) 2005-2011 Timothy D. Morgan
    33 * Copyright (C) 2010 Michael Cohen
    44 * Copyright (C) 2005 Gerald (Jerry) Carter
    … …  
    11521152 * @param file  the file from which key is derived
    11531153 * @param key   the key whose subkey is desired
    1154  * @param name  name of the desired subkey
     1154 * @param name  name of the desired subkey (case-insensitive)
    11551155 * @param index a return value: the index of the desired subkey.
    11561156 *              undefined on error
    … …  
    11711171 * @param file  the file from which key is derived
    11721172 * @param key   the key whose value is desired
    1173  * @param name  name of the desired value
     1173 * @param name  name of the desired value (case-insensitive)
    11741174 * @param index a return value: the index of the desired value. 
    11751175 *              undefined on error
    … …  
    17531753_EXPORT()
    17541754int32_t               regfi_calc_maxsize(REGFI_FILE* file, uint32_t offset);
    1755 int32_t               regfi_conv_charset(const char* input_charset,
    1756                                          const char* output_charset,
    1757                                          uint8_t* input, char* output,
    1758                                          uint32_t input_len, uint32_t output_max);
     1755REGFI_BUFFER          regfi_conv_charset(const char* input_charset, const char* output_charset,
     1756                                         uint8_t* input, uint32_t input_len);
    17591757_EXPORT()
    17601758REGFI_DATA*           regfi_buffer_to_data(REGFI_BUFFER raw_data);

  • trunk/lib/regfi.c

    r261 r262  
    11861186   *      when recovering deleted VK records.
    11871187   */
    1188   int32_t tmp_size;
     1188  REGFI_BUFFER tmp_buf;
    11891189  REGFI_ENCODING from_encoding = (vk->flags & REGFI_VK_FLAG_ASCIINAME)
    11901190    ? REGFI_ENCODING_ASCII : REGFI_ENCODING_UTF16LE;
    … …  
    12001200  else
    12011201  {
    1202     vk->name = talloc_array(vk, char, vk->name_length+1);
    1203     if(vk->name == NULL)
    1204       return;
    1205 
    1206     tmp_size = regfi_conv_charset(regfi_encoding_int2str(from_encoding),
    1207                                   regfi_encoding_int2str(output_encoding),
    1208                                   vk->name_raw, vk->name,
    1209                                   vk->name_length, vk->name_length+1);
    1210     if(tmp_size < 0)
     1202    tmp_buf = regfi_conv_charset(regfi_encoding_int2str(from_encoding),
     1203                                 regfi_encoding_int2str(output_encoding),
     1204                                 vk->name_raw, vk->name_length);
     1205    if(tmp_buf.buf == NULL)
    12111206    {
    12121207      regfi_log_add(REGFI_LOG_WARN, "Error occurred while converting"
    12131208                        " value name to encoding %s.  Error message: %s",
    12141209                        regfi_encoding_int2str(output_encoding),
    1215                         strerror(-tmp_size));
    1216       talloc_free(vk->name);
     1210                        strerror(errno));
    12171211      vk->name = NULL;
     1212    }
     1213    else
     1214    {
     1215      vk->name = (char*)tmp_buf.buf;
     1216      talloc_reparent(NULL, vk, vk->name);
    12181217    }
    12191218  }
    … …  
    12801279   *      when recovering deleted NK records.
    12811280   */
    1282   int32_t tmp_size;
     1281  REGFI_BUFFER tmp_buf;
    12831282  REGFI_ENCODING from_encoding = (nk->flags & REGFI_NK_FLAG_ASCIINAME)
    12841283    ? REGFI_ENCODING_ASCII : REGFI_ENCODING_UTF16LE;
    … …  
    12941293  else
    12951294  {
    1296     nk->name = talloc_array(nk, char, nk->name_length+1);
    1297     if(nk->name == NULL)
    1298       return;
    1299 
    1300     memset(nk->name,0,nk->name_length+1);
    1301 
    1302     tmp_size = regfi_conv_charset(regfi_encoding_int2str(from_encoding),
    1303                                   regfi_encoding_int2str(output_encoding),
    1304                                   nk->name_raw, nk->name,
    1305                                   nk->name_length, nk->name_length+1);
    1306     if(tmp_size < 0)
     1295    tmp_buf = regfi_conv_charset(regfi_encoding_int2str(from_encoding),
     1296                                 regfi_encoding_int2str(output_encoding),
     1297                                 nk->name_raw, nk->name_length);
     1298    if(tmp_buf.buf == NULL)
    13071299    {
    13081300      regfi_log_add(REGFI_LOG_WARN, "Error occurred while converting"
    1309                         " key name to encoding %s.  Error message: %s",
    1310                         regfi_encoding_int2str(output_encoding),
    1311                         strerror(-tmp_size));
    1312       talloc_free(nk->name);
     1301                    " key name to encoding %s.  Error message: %s",
     1302                    regfi_encoding_int2str(output_encoding),
     1303                    strerror(errno));
    13131304      nk->name = NULL;
     1305    }
     1306    else
     1307    {
     1308      nk->name = (char*)tmp_buf.buf;
     1309      talloc_reparent(NULL, nk, nk->name);
    13141310    }
    13151311  }
    … …  
    22432239  REGFI_CLASSNAME* ret_val;
    22442240  uint8_t* raw;
    2245   char* interpreted;
     2241  REGFI_BUFFER tmp_buf;
    22462242  uint32_t offset;
    2247   int32_t conv_size, max_size;
     2243  int32_t max_size;
    22482244  uint16_t parse_length;
    22492245
    … …  
    22762272  talloc_reparent(NULL, ret_val, raw);
    22772273
    2278   interpreted = talloc_array(NULL, char, parse_length);
    2279 
    2280   conv_size = regfi_conv_charset(regfi_encoding_int2str(REGFI_ENCODING_UTF16LE),
    2281                                  regfi_encoding_int2str(file->string_encoding),
    2282                                  raw, interpreted,
    2283                                  parse_length, parse_length);
    2284   if(conv_size < 0)
     2274  tmp_buf = regfi_conv_charset(regfi_encoding_int2str(REGFI_ENCODING_UTF16LE),
     2275                               regfi_encoding_int2str(file->string_encoding),
     2276                               raw, parse_length);
     2277  if(tmp_buf.buf == NULL)
    22852278  {
    22862279    regfi_log_add(REGFI_LOG_WARN, "Error occurred while"
    22872280                  " converting classname to charset %s.  Error message: %s",
    2288                   file->string_encoding, strerror(-conv_size));
    2289     talloc_free(interpreted);
     2281                  file->string_encoding, strerror(errno));
    22902282    ret_val->interpreted = NULL;
    22912283  }
    22922284  else
    22932285  {
    2294     /* XXX: check for NULL return here? */
    2295     interpreted = talloc_realloc(NULL, interpreted, char, conv_size);
    2296     ret_val->interpreted = interpreted;
    2297     talloc_reparent(NULL, ret_val, interpreted);
     2286    ret_val->interpreted = (char*)tmp_buf.buf;
     2287    talloc_reparent(NULL, ret_val, tmp_buf.buf);
    22982288  }
    22992289
    … …  
    23652355    return false;
    23662356
     2357  /* XXX: Should lazily build a hash table in memory to index where keys are when
     2358   *      there are a large number of subkeys.  Attach this to cached keys to
     2359   *      bound the extra amount of memory used.
     2360   */
    23672361  for(i=0; (i < num_subkeys) && (found == false); i++)
    23682362  {
    … …  
    23962390  bool found = false;
    23972391
     2392  /* XXX: Should lazily build a hash table in memory to index where values are when
     2393   *      there are a large number of them.  Attach this to cached keys to
     2394   *      bound the extra amount of memory used.
     2395   */
    23982396  for(i=0; (i < num_values) && (found == false); i++)
    23992397  {
    … …  
    24942492                          uint32_t type, REGFI_DATA* data)
    24952493{
     2494  REGFI_BUFFER tmp_buf;
    24962495  uint8_t** tmp_array;
    2497   uint8_t* tmp_str;
    2498   int32_t tmp_size;
    2499   uint32_t i, j, array_size;
     2496  uint32_t i, j;
    25002497
    25012498  if(data == NULL)
    … …  
    25082505  /* REG_LINK is a symbolic link, stored as a unicode string. */
    25092506  case REG_LINK:
    2510     tmp_str = talloc_array(NULL, uint8_t, data->size);
    2511     if(tmp_str == NULL)
    2512     {
     2507    tmp_buf = regfi_conv_charset(regfi_encoding_int2str(REGFI_ENCODING_UTF16LE),
     2508                                 regfi_encoding_int2str(string_encoding),
     2509                                 data->raw, data->size);
     2510    if(tmp_buf.buf == NULL)
     2511    {
     2512      regfi_log_add(REGFI_LOG_INFO, "Error occurred while"
     2513                    " converting data of type %d to string encoding %d."
     2514                    "  Error message: %s",
     2515                    type, string_encoding, strerror(errno));
    25132516      data->interpreted.string = NULL;
    25142517      data->interpreted_size = 0;
    25152518      return false;
    25162519    }
    2517      
    2518     tmp_size = regfi_conv_charset(regfi_encoding_int2str(REGFI_ENCODING_UTF16LE),
    2519                                   regfi_encoding_int2str(string_encoding),
    2520                                   data->raw, (char*)tmp_str,
    2521                                   data->size, data->size);
    2522     if(tmp_size < 0)
    2523     {
    2524       regfi_log_add(REGFI_LOG_INFO, "Error occurred while"
    2525                     " converting data of type %d to %d.  Error message: %s",
    2526                     type, string_encoding, strerror(-tmp_size));
    2527       talloc_free(tmp_str);
    2528       data->interpreted.string = NULL;
    2529       data->interpreted_size = 0;
    2530       return false;
    2531     }
    2532 
    2533     tmp_str = talloc_realloc(NULL, tmp_str, uint8_t, tmp_size);
    2534     if(tmp_str == NULL)
    2535       return false;
    2536     data->interpreted.string = tmp_str;
    2537     data->interpreted_size = tmp_size;
    2538     talloc_reparent(NULL, data, tmp_str);
     2520
     2521    data->interpreted.string = tmp_buf.buf;
     2522    data->interpreted_size = tmp_buf.len;
     2523    talloc_reparent(NULL, data, tmp_buf.buf);
    25392524    break;
    25402525
    … …  
    25742559   
    25752560  case REG_MULTI_SZ:
    2576     tmp_str = talloc_array(NULL, uint8_t, data->size);
    2577     if(tmp_str == NULL)
    2578     {
     2561    /* Attempt to convert entire string from UTF-16LE to output encoding,
     2562     * then parse and quote fields individually.
     2563     */
     2564    tmp_buf = regfi_conv_charset(regfi_encoding_int2str(REGFI_ENCODING_UTF16LE),
     2565                                 regfi_encoding_int2str(string_encoding),
     2566                                 data->raw, data->size);
     2567    if(tmp_buf.buf == NULL)
     2568    {
     2569      regfi_log_add(REGFI_LOG_INFO, "Error occurred while"
     2570                    " converting data of type %d to string encoding %d."
     2571                    "  Error message: %s",
     2572                    type, string_encoding, strerror(errno));
    25792573      data->interpreted.multiple_string = NULL;
    25802574      data->interpreted_size = 0;
    … …  
    25822576    }
    25832577
    2584     /* Attempt to convert entire string from UTF-16LE to output encoding,
    2585      * then parse and quote fields individually.
    2586      */
    2587     tmp_size = regfi_conv_charset(regfi_encoding_int2str(REGFI_ENCODING_UTF16LE),
    2588                                   regfi_encoding_int2str(string_encoding),
    2589                                   data->raw, (char*)tmp_str,
    2590                                   data->size, data->size);
    2591     if(tmp_size < 0)
    2592     {
    2593       regfi_log_add(REGFI_LOG_INFO, "Error occurred while"
    2594                     " converting data of type %d to %s.  Error message: %s",
    2595                     type, string_encoding, strerror(-tmp_size));
    2596       talloc_free(tmp_str);
    2597       data->interpreted.multiple_string = NULL;
    2598       data->interpreted_size = 0;
    2599       return false;
    2600     }
    2601 
    2602     array_size = tmp_size+1;
    2603     tmp_array = talloc_array(NULL, uint8_t*, array_size);
     2578    tmp_array = talloc_array(NULL, uint8_t*, tmp_buf.len+1);
    26042579    if(tmp_array == NULL)
    26052580    {
    2606       talloc_free(tmp_str);
     2581      talloc_free(tmp_buf.buf);
    26072582      data->interpreted.string = NULL;
    26082583      data->interpreted_size = 0;
    26092584      return false;
    26102585    }
    2611    
    2612     tmp_array[0] = tmp_str;
    2613     for(i=0,j=1; i < tmp_size && j < array_size-1; i++)
    2614     {
    2615       if(tmp_str[i] == '\0' && (i+1 < tmp_size) && tmp_str[i+1] != '\0')
    2616         tmp_array[j++] = tmp_str+i+1;
     2586
     2587    tmp_array[0] = tmp_buf.buf;
     2588    for(i=0,j=1; i < tmp_buf.len && j < tmp_buf.len; i++)
     2589    {
     2590      if(tmp_buf.buf[i] == '\0' && (i+1 < tmp_buf.len)
     2591         && tmp_buf.buf[i+1] != '\0')
     2592        tmp_array[j++] = tmp_buf.buf+i+1;
    26172593    }
    26182594    tmp_array[j] = NULL;
    … …  
    26202596    data->interpreted.multiple_string = tmp_array;
    26212597    /* XXX: how meaningful is this?  should we store number of strings instead? */
    2622     data->interpreted_size = tmp_size;
    2623     talloc_reparent(NULL, tmp_array, tmp_str);
     2598    data->interpreted_size = tmp_buf.len;
     2599    talloc_reparent(NULL, tmp_array, tmp_buf.buf);
    26242600    talloc_reparent(NULL, data, tmp_array);
    26252601    break;
    … …  
    26632639
    26642640/******************************************************************************
    2665  * Convert from UTF-16LE to specified character set.
    2666  * On error, returns a negative errno code.
     2641 * Convert string from input_charset to output_charset.
     2642 * On error, returns a NULL buf attribute and sets the errno.
    26672643 *****************************************************************************/
    2668 int32_t regfi_conv_charset(const char* input_charset, const char* output_charset,
    2669                            uint8_t* input, char* output,
    2670                            uint32_t input_len, uint32_t output_max)
     2644REGFI_BUFFER regfi_conv_charset(const char* input_charset, const char* output_charset,
     2645                                uint8_t* input, uint32_t input_len)
    26712646{
    26722647  iconv_t conv_desc;
    26732648  char* inbuf = (char*)input;
    2674   char* outbuf = output;
    2675   size_t in_len = (size_t)input_len;
    2676   size_t out_len = (size_t)(output_max-1);
     2649  char* outbuf;
     2650  char* retbuf;
     2651  size_t allocated = (size_t)input_len;
     2652  size_t in_left = (size_t)input_len;
     2653  size_t out_left = (size_t)allocated-1;
     2654  REGFI_BUFFER ret_val;
    26772655  int ret;
    26782656
     2657  ret_val.buf = NULL;
     2658  ret_val.len = 0;
     2659  retbuf = talloc_array(NULL, char, allocated);
     2660  outbuf = retbuf;
     2661  if(outbuf == NULL)
     2662  {
     2663    errno = ENOMEM;
     2664    return ret_val;
     2665  }
     2666
     2667  /* Set up conversion descriptor. */
    26792668  /* XXX: Consider creating a couple of conversion descriptors earlier,
    26802669   *      storing them on an iterator so they don't have to be recreated
    26812670   *      each time.
    26822671   */
    2683 
    2684   /* Set up conversion descriptor. */
    26852672  conv_desc = iconv_open(output_charset, input_charset);
    26862673
    2687   ret = iconv(conv_desc, &inbuf, &in_len, &outbuf, &out_len);
     2674  ret = 0;
     2675  do
     2676  {
     2677    if(ret == -1)
     2678    {
     2679      retbuf = talloc_realloc(NULL, retbuf, char, allocated+(in_left*2));
     2680      if(retbuf == NULL)
     2681      {
     2682        errno = ENOMEM;
     2683        return ret_val;
     2684      }
     2685      outbuf = retbuf+(allocated-1-out_left);
     2686      out_left += in_left*2;
     2687      allocated += in_left*2;
     2688    }
     2689    ret = iconv(conv_desc, &inbuf, &in_left, &outbuf, &out_left);
     2690   
     2691  } while(ret == -1 && errno == E2BIG);
     2692 
    26882693  if(ret == -1)
    26892694  {
    26902695    iconv_close(conv_desc);
    2691     return -errno;
    2692   }
    2693   *outbuf = '\0';
    2694 
    2695   iconv_close(conv_desc); 
    2696   return output_max-out_len-1;
    2697 }
    2698 
     2696    return ret_val;
     2697  }
     2698
     2699  /* Save memory */
     2700  if(out_left > 0)
     2701  {
     2702    retbuf = talloc_realloc(NULL, retbuf, char, allocated-out_left);
     2703    if(retbuf == NULL)
     2704    {
     2705      errno = ENOMEM;
     2706      return ret_val;
     2707    }
     2708    allocated -= out_left;
     2709  }
     2710  retbuf[allocated-1] = '\0';
     2711  iconv_close(conv_desc);
     2712
     2713  ret_val.buf = (uint8_t*)retbuf;
     2714  ret_val.len = allocated-1;
     2715  return ret_val;
     2716}
    26992717
    27002718

  • trunk/python/pyregfi/__init__.py

    r261 r262  
    106106# @note Developers strive to make pyregfi thread-safe.
    107107#
    108 # @note Key and Value names are case-sensitive in regfi and pyregfi
    109 #
    110108import sys
    111109import time
    … …  
    390388    #
    391389    # @param name The name of the subkey or value desired. 
    392     #             This is case-sensitive.
    393     #
    394     # @note The registry format does inherently prevent multiple
    395     #       subkeys or values from having the same name. 
    396     #       This interface simply returns the first match. 
     390    #             This is case-insensitive.
     391    #
     392    # @note The registry format does not inherently prevent multiple
     393    #       subkeys or values from having the same name, having a key
     394    #       and a value with the same name, or having the same name in
     395    #       different cases that could both match.
     396    #       This interface simply returns the first match in the list.
    397397    #       Lookups using this method could also fail due to incorrectly
    398     #       encoded strings.
    399     #       To identify any duplicates, use the iterator interface to
    400     #       check every list element.
     398    #       encoded strings stored as names.
     399    #       To identify any duplicates or elements with malformed names,
     400    #       use the iterator interface to check every list element.
    401401    #
    402402    # @return the first element whose name matches, or None if the element
Note: See TracChangeset for help on using the changeset viewer.