Hey everyone,

I want to debug a Python Skript using a different arch (x86_64 instead of arm64) then my default terminal. I was looking for ways to define a terminal profile (see also on stackoverflow) using the launch.json, but I haven't found a way yet. I started trying to use the argument preLaunchTask, but this seems to be executed outside of the terminal (and it also freezes for this specific task). Any ideas how to fix this?

// launch.json
{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python (x86_64): script.py",
            "type": "python",
            "request": "launch",
            "program": "/path/to/script.py",
            "console": "integratedTerminal",
            "justMyCode": true,
            "preLaunchTask": "x86_64"
        }
    ]
}

//tasks.json
{
    "version": "2.0.0",
    "tasks": [
        {
            "label": "x86_64",
            "type": "shell",
            "command": "arch -x86_64 zsh"
        }
    ]
}

I'm engaged in a scraping project, and I've been struggling to get JSON data out of script tags. BeautifulSoup is getting the script tag and its contents just fine, but I'm uncertain of anything that works to parse the JSON out cleanly. Using regex has not been super effective, and I've run into problems with it. Based on what I could find, that's probably more an issue with my regex knowledge.

I coded up the below based on the idea that I can step through a string, index the point when a JSON block probably starts, count the degree of nesting (with the depth variable) and index the point when I come out of the block when depth returns to 0, then pass this slice to json.loads. The output is then passed to a list and returned.

It seems to work based on limited testing, but I don't want to abuse the website just to test if this will work effectively. I feel happy that I've gotten this far, but my use of the first few if/elif statements feels janky and crap, and the size of the JSON blocks are just too huge to really read manually to check if I'm getting anything wrong.

My questions are:

• is there a better way to manage the flow control with the first few while/if/elif statements?

• is there a library that will do this for me?

• would regex actually be easier, and if so, what would work?

• am I likely to run into any data parsing problems with the below code?

Notes on the scraped website: the website often uses <script> to hold JSON data, or a mix of JS and JSON, sometimes just JS, sometimes it holds a src link to another page with no other content. The JSON itself is properly formatted, but it may not sit in the script tag cleanly, and sometimes there are multiple blocks with non-JSON content in between. There will be some ~15 script tags per page, but this varies. The script content will have a character length of 100000 on average, but ranges between 0-200000. Sometimes they are tagged appropriately as JSON data, but often not.

def JSON_extr(incoming):
    # various pointers
    index = 0 # will act as starting point for incrementing the string.find function
    depth = 0 # will track the degree of depth
    pointer = 0 # pointer for tracking location w/in string segment
    first = True # switches to False while inside a JSON block
    json_lists = []
    finalising = False # sets to True when no {" remain
    
    while (('{"' in incoming[index:]) or ('}' in incoming[index:])): 
        # while there is something left to find
        brack_open = incoming[index:].find('{"')
        brack_clos = incoming[index:].find('}')
        
        if not ('{"' in incoming[index:]): 
            # sets later functions to resolve remaining 
            finalising = True
            
        if -1 < brack_open < brack_clos: 
            # negative will indicate none left, and cause a bug
            pointer = brack_open
            depth +=1
        elif ((brack_clos < brack_open) or finalising): 
            # finalising kicks if there are no {"s left;
            # otherwise brack_clos will always be lower
            pointer = brack_clos
            depth -=1
            
        if first == True: # if starting a JSON block, keep a note of the index
            first = False
            start = pointer+index
            
        if depth <= 0: # if we return out of the block, assess data as JSON
            packet = incoming[start:(index+pointer+1)]
            if packet:
                try: # try to convert to json; if it fails, report warnings
                    json_lists.append(json.loads(packet))
                except json.JSONDecodeError as e:
                    if start == index+pointer:
                        pass # empty parentheses
                    else:
                        warnings.warn(f"Possible JSON candidate between indexes {start}:{index+pointer+1} failed with {e}")
            first = True # reset the possibility of another JSON block
            if depth <0:
                warnings.warn(f'Nesting error in found, resetting nesting depth to zero.')
                depth = 0
        index +=(pointer+1)
        counter +=1
    return json_lists

Really appreciate any help with this!

I am using Watson assistant for a chatbot and there is a machine Learning code that I want to run in the backend. The user input is present in the JS. I need to to able to send this input to my python script to run the algorithm and send the output to the user.

Pardon if what I said doesn't make complete sense. I'll be available for providing any clarifications.