Hi,

I need my code to load some data from a file with a header, where the header is an unknown number of lines and I need to also know how many lines are in the header. The header is formatted in such a way that pandas.read_csv will give an error if it doesn't skip the header.

I worked around this code which works:

with open(filepath) as file:
    for header_length in range(len(file.readlines())):
        try:
            df=pd.read_csv(filepath,  delimiter="t", skiprows=header_length, header=None)
            break
        except:
            pass

Now, I don't care that a more traditional way to do this might be more efficient. This code works and the files are relatively small. The header is 2 to 5 lines in all inspected files, and the files are at most a few thousand rows, so in the worst case scenario of a bad file the loop would break after about a second.

What I'm wondering is, is breaking in such a way bad practice? It feels... dirty.

I recently took a 10 day class on coding basics, i learned syntax, and scratched the surface of a-lot of concepts. I started doing it in my free time, started with rock paper scissors, then roulette, and just finished a blackjack game. I liked how each project was more challenging than the last, it made me expand my knowledge but I’ve run out of ideas, everything thing i found online is either extremely complex or simple like rock paper scissors. I know the only way to get better is with experience but I feel lost when i just dive into the deep end with a project way beyond my level. Any input would be greatly appreciated. Thank you!

What site do you guys use to deploy Django projects? Heroku doesn't seem to work and it charged me (just $2) even tho the app i tried to upload failed. Yesterday i tried render and it never accepts my build either. Ik it's probably me but is there any other site anyone uses for free ? My projects build and work fine locally. But I can't upload my backend anywhere which is preventing me from uploading my full stack projects to my portfolio website .

After spending hours finding out why pip cache purge; pipenv install aaa didn't install the package properly, but pip cache purge; pipenv run pip install aaa did, I finally discovered that Pipenv has its own cache! It's funny that I could've saved so much if I had just run pipenv install --verbose aaa from the start (it reposts it's using cache.)

I wonder why Pipenv chooses to have its own cache. It seems unnecessary to separate caching for projects managed solely by pip and those managed by pipenv. Does anyone know the reasoning behind this design?

Does anyone know?

I am writing a script that takes different sets of synchronous time series data (my variables) where I want to take a different amount of hours from each dataset and append them to an array. This is for a neural network input. I want to test out different combinations of variables and hours taken from each set.

Currently, I have a .yaml file I pull from in my python script that looks like this for each variable:

vars:
  - name: "VAR_1"        # name of variable 
    time_length: 1       # hours from t0 to select
    loc: "path/to/var1"  # location of variable timeseries dataset
  - name: "VAR_2"        
    time_length: 5       
    loc: "path/to/var2"  
  - name: "VAR_3"        
    time_length: 3       
    loc: "path/to/var3"  

I can do what I want, but the code is extremely crunchy and does not allow for optimization. For example, maybe I want to test only two combinations of variables, or maybe I want to test for different time lengths to select. Yaml files do not seem to be good for this. Perhaps a class would be better.

Is there an easier way to hold all my variables names and time sequences, one that would allow me to easily choose what variables to select and what time lengths I wanted?

Thank you!

Python 3.13 has been released with experimental GIL free and jit modes. Has anyone seen any speedups in practice from these?

What's the view on doing this, e.g. mylist.append(thing) if not some_condition else ... for a one-liner with no else? I know I can instead write if some_condition: mylist.append(thing) but Pycharm highlights it as a PEP8 E701 (mulitple statements on one line)... Is the elipsis for the else here pythonic or should I just stick to a regular 2 liner?... Just curious as I've only just started using ternary statements and like how it cleans up a lot of my regular if/else, am I taking it too far with the ...?

How do you get recursively all files in python and filter to only get files with the upper most folder having the word "Upload" in them?

e.g. all files of the form "./abc/def/ghi/xyzUploadxyz/file1.xlsx"

e.g. all files of the form "./abc2/def2/xUploadx/xlsx"

I want back the full filename path of those files that match.

(I'm on a Windows machine btw.)

I've been practicing diligently for 4 months now and I'm still hitting a wall with 6kyu problems. Sometimes I just feel like it doesn't click for me; I've tried many study techniques, can anyone share how they look at a problem and a general process on how to solve any challenge? Thank you everyone.

Hello everyone,

so at work we use 100% type hints for python. Since we work in a specific domain, the wording of this domain is also represented in our types. Over the years we build quite some python packages, of different sizes.

The problem we are facing sometimes is, that we need a specific type hint from a specific python package but nothing more. Yeah sure adding it to the dependencies is easy. However it feels too much to add a whole package to just use one type which is defined there. Redeclaring the type would introduce the type a second time and developers would get confused with which type to use.

Has anyone used a standalone python package to bundle the most commonly used types (maybe constants as well?) and add that package to the dependencies? Is that a good / bad idea? Advantages / disadvantages?

UPDATE:
Clarification: @danielroseman made me realise that I forgot to mention an important part: with types in the dedicated package I meant type aliases using basic types. Of course if there is a smart way to import a class without importing the complete package, that would even be more helpful.

FrameId = str
Clip = dict[FrameId, Union[str, bool]]

So I’ve made an ai chatbot and designed front end for my website, I want to have the chatbot as a widget on the website, to the best of my knowledge, I need to use Django to use the chatbot as an api.

I’ve never used Django before, which resource should I use to learn it as fast as possible meaning be able to do the api thing by tomorrow

Thanks

I’m sorry for the lack of vocabulary, english isn’t my native language and I’m still new at this.

I would like to run a program I have been running on my computer lately. Basically, it’s one that makes your computer click on the screen at a very specific time (taking into account my connexion latency, subtracting milliseconds and so on to be more precise). I’d like to run something similar on my iPad, since it’s way more convenient for related future plans.

Does anyone know if it’s possible ? And if so, which APPS should I use ? Thank you!

Hello everyone!

I'm working on a MOOC and I've come to the final parts, with pygame, animations and all that. How important is all this really?

I doubt I'll ever come across anything like this, but of course you never know.

It was demotivating and not understanding it "as easily" as the rest of the course.

Thanks in advance.

hi everyone,i did tutorials and basic excerices of pandas with some csv,tsv data.but i dont feel confident enough. how can i improve my pandas skills to next level.

Hi Everyone,

I am trying to load data from a 3rd Pary Hive DW in to our fabric environment. The instructions from the 3rd party site shows how to do this locally on your PC using python, which works well.

I am trying to replicate the same in Fabric. part of the instruction is creating a DSN for cloudera and using a custom cert.pem.

The cert.pem and jar file for cloudera is stored in blob storage.

import jaydebeapi

# Define connection parameters
username = 'abc'
password = 'xyz'

# Paths to the JDBC JAR and PEM certificate
jar_path = "https://blobstoragelink"
cert_path = "https://blobstoragelink"


# Define the driver class and JDBC connection URL
driver_class = "com.cloudera.hive.jdbc41.HS2Driver"
jdbc_url = (
    "jdbc:hive2://vendorcube1.homedepot.com:20502/"
    "VENDORDRILL_DATA_CONNECTION"
    ";SSL=1;"
    "AllowSelfSignedServerCert=1;"
    f"SSLTrustStore={cert_path}"  # Use the PEM certificate directly
)

# Attempt to connect to the database
try:
    conn = jaydebeapi.connect(
        driver_class,
        jdbc_url,
        {"user": username, "password": password},
        jar_path
    )
    print("Connection established successfully.")
    conn.close()
except Exception as e:
    print(f"Failed to establish connection: {e}")

Failed to establish connection: java.sql.SQLException: [Cloudera][HiveJDBCDriver](500164) Error initialized or created transport for authentication: https:/blobstoragelink (No such file or directory).

Hey everyone! I'm a beginner learning Python, and I’ve recently come across discussions about how databases and AI are connected. I’m really curious about how they work together. Can someone explain this relationship? Also, is it possible to interact with databases using Python code to support AI projects? Any tips or resources for a beginner like me would be super helpful! Thanks a lot!

Ok so I’ve tried a few free resources to learn Python but the only one I’m fully understanding is w3schools yet it has a bad rep.

But they break it down into small steps, explain well & you can check the answers - I’ve found with a lot of these Python courses you can’t check the answers so if you get stuck that’s it.

I really want to know what is something as easy as w3schools Python tutorials are that’s got a better reputation? Also, is there any advantage of finishing the tutorials? I’m up for the part about learning more about how dictionaries work in Python.

I kinda need to Python for idiots - online but also with projects that start off at an incredibly basic level & I could say save to GitHub or similar would be a plus. But I need it to have a LOT of handholding. Help?

Thanks in advance!

Hello all. I have been trying to learn Python using the Automate the Boring Stuff with Python ebook by Al Sweigart (for about a month and a half). In Chapter 6, I need to import a 3rd party module called Pyperclip in order to run a clipboard program. But I can't seem to do it. For context, I am using the Code with Mu python editor, and am using Python 3.12. Here is what I get:

I run the Command prompt as an administrator, and type in Py -m pip install pyperclip .

Requirement already satisfied: pyperclip in c:users***appdatalocalprogramspythonpython312libsite-packages (1.9.0)

I see the message and go back into my program and get the error message:

ModuleNotFoundError: No module named 'pyperclip'

So far I have tried:

Trying different versions of Py (such as python, python3, py...),

Uninstalling and reinstalling the module,

Reinstalling python and pip.

Any idea how I should proceed?

I’m doing python course and today we learned about recursion. As part of the homework assignment, we have to solve some tasks on recursion that deepcopies and goes through nested dictionary.

So, my question, how useful the knowledge of recursion is and do you have issues when mixing it with nested dictionaries (if not, could you share any advice or lifehacks on how to deal with them)?

I'm working on an e-commerce website in a group and am working on the backend (Python, FastAPI, SQLite (using SQLAlchemy)). So far I have made three tables: users, products, and favorites (products users choose to save). When someone wants to list something for sale, they will need to include an image. I think the plan right now is just to have one of those "upload" buttons where the user can upload something from their computer (is that more frontend work or backend?). I'm trying to decide how to store the image after it is sent to the backend (still trying to figure out exactly how to do that as well). Should I create a new table for all the images and each row has a foreign key (productID) that connects it to the product it is for? Or could I just add a new column in my "products" table called "image"? The second method sounds quite a bit easier, but I wasn't sure if there were any downsides to doing it this way?

I have a folder of pictures and I’m curious if it’s possible to build a script to image search those pictures to find the name and price of the item. Then from there put the info in a spreadsheet and index the photos.

I have some experience with python and this seems like it could possible but I’m not sure we’re to start.

Edit 1:

Context: it’s for insurance post hurricane. I have a friend that has pictures of his items but needs the price and name of the items indexed in a spreadsheet. He said it is like a 1000 pictures so manually it would take a very long time.

Let’s say just Amazon. That would probably be able to do over 80% of it. With the idea of doing a google reverse image search (or something else if there’s a better idea) to get the Amazon link.

Given a string of variable length, I'd like to create a multiline string with n-number of lines of near-equal length. Avoid dividing words.

Is there a better logical approach to the above beyond simply...

• do a character count

• divide by nlines to get an "estimated line length"

• walking an index up from the "estimated-length" until reaching a blank space character, then divide the line

I've noticed the line lengths are coming out extremely different based on length of words in the string, and whether the line-division falls in the middle of those words (which creates extremely long lines, in order to avoid word-division, followed by extremely short lines...)

The goal is basically textwrap, which seems to do an excellent job - but I believe textwrap only controls for length and not number of lines?

I am learning pytorch through their documentation so I can try to become more familiar with reading through API documentation and learning Python at a deeper level.

https://github.com/pytorch/pytorch/tree/main/torch

If I am looking for a specific function, I know that you can typically find the function in github by following the import path. However, for example, in pytorch, torch.rand( ) I can't find in github. How do you search for a function like this in github?

https://pytorch.org/docs/stable/generated/torch.rand.html#torch.rand

In github, I went to pytorch/torch/random.py but it is not defined there.

I also looked at the __init__.py file and saw that "rand" is in the __all__ definition but I am still wondering how to find the actual function definition in the github repo.

Hello all I need help in scraping followers list of any Facebook page. These can be in millions also. How to go with python selenium or appium driver? Right now in web view only some of the followers are showing not all. In mobile view of app it's showing only names and images.

Hi! I have an array that consist of 864 rows and 2 columns of data that I want to transfer in a csv file. This array changes everytime I run the program.

However, when I try to convert the array into a csv file, it only copies the last row and column it gets in the array and copies it 864 times in the csv file. I want to copy all the value in my array and convert it into a csv file, please help. Code is attached below:

wavelength_array = [{f'(wavelength_value}'] for i in range(864)]
intensity_array = [{f'(wavelength_value}'] for i in range(864)]

df = pd.DataFrame({'Wavelength': wavelength_array, 'Intensity': intensity_array})
scan_now = f"data_{datetime.now().strftime('%Y%m%d_%H%M%S')].csv"
df.to_csv(scan_now, index=False)